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ABSTRACT 



A surgical manipulator system comprising at least one 
surgical manipulator, at least one gmde, a detector, and a 
drive controller. Ihe surgical man^ulator bas a sucgical 
device for performing a desired operation. The guide guides 
the surgical device. The detector detects a position and/or 
orientation relationship between the suigical device and the 
guide, and/or a position and/or orientation relationship 
between the surgical device and another surgical device. The 
drive controller controls the surgical manipulator such that 
the surgical device is guided by the guide. 

18 Oaims, 45 Drawing Sheets 
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SURGICAL MANIPULATION SYSTEM 

This application is a Continuation of application Ser. No. 
08/316^33. filed Sep. 30, 1994. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a surgical manq)ulator 
system, which ihchides a manipulator to be inserted into a 
body cavity of a subject and v^cb is designed to remote- 
control the manq>ulator to make diagnosis or perfomi a 
surgery. 

2. Description of the Related Art 

In recent years, transcutaneous endoscope surgery has 
been performed in which an endoscope or a medical instru- 
ment is inserted into a body cavity through a bole cut in the 
body wall (e.g., the abdominal wall) to perform various 
treatments in the body cavity. Hiis is because transcutaneous 
endoscope surgery involves no large-scale incision and is 
therefore scarcely invasive to the patient. This type of 
surgery is now widely performed to extract the gallbladder 
or a part of either hing. 

To perform a successful transcutaneous endoscope 
surgery, it is desirable that the endoscope or the instrument 
inserted in the body cavity be manipulated in as broad a 
space as possible within the body cavity. An endoscope or an 
instrument a surgeon can manipulate with one hand is a 
straight one which has but Httle freedom of operation. Due 
to the little freedom of operation, it is difficult for a surgeon 
to orient the endoscope or instrument in order to observe the 
target object at a desired angle or to treat the target object, 
even if he or she has managed to guide the endoscope or 
instrument to the object in the body cavity. For example, the 
surgeon can hardly position an instrument holding a needle, 
so as to drive the needle perpendicular to a suture line in his 
or her effort to suture an organ located in the body cavity. 

Hus problem with transcutaneous endoscope surgery is 
solved by the use of a suigical manipulator with a multi-joint 
insertion section wtiich has great operation freedom. A 
surgical manipulator system which holds an endoscope or a 
medical instrument and whic^ is manipulated by a surgeon 
is disclosed in U.S. Pat. No. 5,217,003. This system com- 
prises a manipulator ^ich has an actuator and a multi-joint 
insertion section for holding an endoscope or an instrument. 
The actuator drives the joints of the insertion section 
independently, v^ereby the insertion section is easily guided 
to a target organ located in a body cavity. 

When the multi-joint insertion section is operated to 
orientate the endoscope or instrument at a desired angle to 
the target organ, some of the joints may contact another 
organ in the body cavity. Further, when the patient moves 
wliile undergoing transcutaneous endoscope surgery, the 
affected part moves, too, making it dif&cult for the surgeon 
to excise that part precisely. 

TWo types of surgical manipulator systems are available, 
each comprising a remote-control device, a manipulator and 
a TV monitor. The TV monitor di^lays an image of the 
affected part in a body cavity, which has been obtained 
through an endoscope inserted in the body cavity. In the 
system of the first type, the surgeon operates a joy stick used 
as the remote-control device, while looking at the image of 
the affected part on the TV monitor, thereby operating the 
manipulator in the same way in the body cavity. In the 
system of the second type, generally known as "master-slave 
system," the surgeon operates a master manipulator used as 



6,325 

2 

the remote-control device, while looking at the image on the 
TV monitor, thereby operating the slave manipulator in the 
same maimer in the body cavity. 
In either system, the remote-control device is physically 

^ separated from the manipulator. Therefore, the directions in 
which the remote-control device and the manqiulator may 
move are determined by the positional relation in which the 
device and the manipulator are arrajiged. In some cases, the 
coordinate system for tbe manipulator may be orientated 

^0 differently from the coordinate system for the remote- 
control device. 

In the case of a surgical manipulator system which 
comprises a plurality of master manq)ulators and a plurality 
of slave manipulators, the master manipulators are usually 
located at the bedside close to the surgeon so that he or she 
can operate them, whereas the slave manipulators are 
arranged opposite the bedside since no ^aoe is available at 
any other bedside. To state another way, the master manipu- 
lators and the slave manipulators cannot be located at the 

^ same bedside in most cases. As a consequrace, when the 
surgeon moves the corresponding master manipulator to the 
right in order to move the slave maninilator in the same 
direction, a slave manipulator may move to the left. In other 
words, the coordinate system for the slave manipulators is 

^ orientated differently from that for the master maDi)ulators, 
increasing the possibility that a slave manqnilator moves in 
a direction other than the very direction in which it should 
move as the surgeon desires. 

Even if the slave manipulators are arranged at the same 
bedside as the master manipulators, they should be spaced 
away from the master manipulators so as not to interfere 
with the master manipulators. Hence, the slave manipulators 
must be orientated differently from the master manipulators. 
In this case, too, any slave manipulator may move in a 
direction other than the direction in which it should move as 
the surgeon desires. 

When manipulators each holding an endoscope or an 
instrument are independently operated by using rcmote- 

^ control devices, to thereby perform surgery, each manipu- 
lator is moved in the direction which is determined by the 
positional relationship between the remote-control devices, 
the positional relation betweoi the manipuIatOFS and the 
positional relation between each manipulator and the asso- 

45 ciated remote-control device. That is, tbe direction in wiiidi 
each manipulator can be moved is restricted. In some case, 
the manipulator may be moved in an urulesirable direction. 

In the master-slave manipulator system, so-called "bilat- 
eral contror' is employed to provide a surgeon a sense of 

50 immediacy. Bilateral control consists in transmitting any 
force on the slave manipulator to the master manipulator and 
hence to the person operating the master manipulator. To 
enhance visual sense of immediacy, a 3D (3-dimensional) 
image of the object being treated is displayed on a TV 

55 monitor installed near the master manipulator. 

Since the master manipulator is located far from the 
object, it cannot abut on the object no matter how the 
surgeon operates the master manipulator. Hence, the surgeon 
is likely to operate the master manipulator excessively, 

60 whereby the slave manipulator collides with the object 
excessively. If this happens, an excessive force is exerted on 
cither the object or the slave manipulator, possibly damaging 
the object or tbe slave manipulator. To prevent such damage, 
the surgeon may operate the master manipulator to move the 

65 master manipulator away from the object as soon as he or 
she sees that the slave manipulator gets too close to the 
object. His or her action may be too late. This is because a 
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long lime is required to process signals generated by the The fourth object of the invention is to provide a surgical 

master manipulator and to supply the signals to the slave manipulator system with whidi it is possible to locate an 

manipulator. object correctly and treat the object within a short time. 

The surgeon may manipulate the master manipulator The fifth object of this invention is to provide a surgical 

without seeing the 3D image displayed on the TV monitor s manipulator system in which an endoscope and a medical 

to give him or her visual s^ise of immediacy. In this case, instrument can be moved so freely that the interior of an 

the slave manipulator is likely to collide with the object, body cavity may be observed and an object in the body 

applying an excessive force on the object or receive an cavity may be treated, and are prevented to contact an organ 

excessive reaction from the object. other than the target one to apply an excessive force thereon. 

Generally, the insertion section of a surgical manipulator The sixth object of the present invention is to provide a 

is integral with an endoscope or a medical instrument. surgical manipulator system, in which only devices (e.g., an 

Neither the endoscope nor tbc medical instrument can be endoscope and a medical instrument) to be inserted into a 

washed or sterilized, indepeixiently of the surgical manipu- body cavity or used and removed from a body cavity, can be 

lator. To wash and sterilize the endoscope or instrument, it sterilized and washed, and problems restricting the operation 

is necessary to wash and sterilize the manq)ulator, as well. of such devices can be solved easily and effectively. 

It would t^ much labor to wash and sterilize the manipu- The seventh object of this invention is to provide a 

lator which is large and massive. surgical manipulator system which has a master manipulator 

When a manipulator having a medical instrument and a and a slave man^ulator and with which a surgeon can 

manipulator having an endoscope arc simultaneously perfonnanemeigency treatment immediately after he or she 

operated, they may interfere with each other during the ^ stops remote-controlling the slave manipulator, rendering it 

surgical operation. If this takes place, the ^aces in which the unnecessary for another surgeon to perform the emergent 

manipulators can move will be limited, inevitably reducing treatment and thus redudng the surgical cost 

the operation fi^dom of both the medical instrument and To attain these objects, the following surgical manipulator 

the endoscope, and ultimately impairing the operability of systems arc provided according to the present invention, 

the surgical man^ulator system as a whole, lb make matters a first surgical manipulator system according to the 

worse, the instrument and the endoscope caimot be invention comprises: at least one surgical manq;)ulator hav- 

exchanged to assume each other's position since they are j^g surgical means for performiog a desired operation; at 

integral with the insertion sections of the respective manipu- least one guide means for guiding the surgkal means; 

lators. To release the manipulators from the mutual detecting means for detecting a poatbn and/or orientation 

interference, it is necessary to remove the insertion sectioits relationshq) between the surgical means and the guide 

from the body cavity, then to position the manipulators out means, and/or a position and/or orientation re]ationdiq» 

of mutual interference, and finally to insert the insertion between the surgical means and another surgical means; and 

sections back into the body cavity. Ibis sequence of opera- drive control means for controlling the surgical qaa^iipilator 

tions is complex and time-consuming; it not only decreases such that the surgical means is guided by the guide means, 

the efficiency of the surgical operation, but also impose a x second surgical manipulator system according to the 

great burden on the patient receiving the operation. invention is designed to achieve the first, second and fourth 

In the master-slave manipulator system, described above, objects. In this system, the detecting means has a first 

the surgeon operates the master manq>ulator at a place decision means for determining a position and/or orientation 

remote from the slave manipulator and the patient, and ^ of the guide means by using a specific coordinate system as 

therefore caimot attend the patient immediately in case the a referexKx, a second dedsion means for determining a 

patient gets, for example, a heart attack, or falls into a critical position and/or orientation of the surgical manipulator by 

condition. Nor can he or she attend the patient at once incase using a specific coordinate system as a reference, and a 

the patient suffers great bleeding which caimot be stopped by coordinate-system switching means for switching the coor- 

remote-controlling the slave manipulator, and the abdominal dinate system of at least one of the guide means and the 

wall must be immediately incised in order to stop the suxgical manipulator, to another, thereby to set the guide 

bleeding. Another surgeon needs to stay by the patient to means and the surgical manipulator in a specific relationship 

perform an emergency treatment such as heart massage, in terms of position and/or orientation, 

incision of the abdominal wall, or the like. a third surgical manipulator system according to the 

<;timmapy np thf intvfntion 50 invention is designed to accomplish the fourth object in 

SUMMARY OF THE INVENTION particular, and characterized in that the detecting means 

The first object of the present invention is to provide a comprises a permanent magnet attached to the first surgical 

surgical manipulator system which comprises a manipulator manipulator and a Hall element attached to the second 

and a remote-control device for controlling the manipulator surgical manipulator. 

and in which the manipulator can be moved in a desired 55 a fourth surgical manipulator system of the invention is 

direction, not restricted by the positional relation between it designed to readily attain the third object. This system 

and the remote-control device. comprises, among other things, two guide means are pro- 

The second object of the invention is to provide a surgical vided and operation means for calculating tool center points 

manipulator system which makes a surgeon feel as he or she which the two guide means are to assume. The surgical 

were in a body cavity, and with which he or she perform a eo manipulator is controlled, moving the tool center points of 

transcutaneous endoscope surgery in the same way as an the surgical means to positions indicated by the guide 

abdominal operation. means, only when the difference between the position of the 

The third object of this invention is to provide a surgical tool center point of each surgical means and the position of 

manipulator system which has a master manipulator and a the tool center point calculated by the operation means falls 

slave manipulator, and in which no excessive force is 65 within a predetermined range. 

exerted on an object or the slave manipulator even if the A fifth surgical manipulator system according to this 

master manipulator is operated in an erroneous manner. invention is designed to achieve the fifth object In the 
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system, the surgical means comprises a straight insertion 
section and a surgical section which is connected to a distal 
end of the insertion section and which is able to bend to treat 
an affected part of a subject and/or provide an image of the 
affected part of die subject, and the surgical manipulator s 
comprises coupling means for holding the insertion section, 
while allowing the insertion section to move back and forth, 
and positioning means for positioning the insertion section. 

A sixth surgical manipulator system of the invention is 
designed to attain the sixth object, and comprises, among lo 
other things, a connecting mechanism for removably con- 
necting at least one part of the surgical means to the surgical 
manipulator. 

A seventh surgical manipulator system according to the 
present invention is designed to accomplish the sevratb 15 
object, and comprises, among other things, fastening means 
for removably fastening the surgical manipulator to an 
operating table. 

Additional objects and advantages of the invention will be 
set forth in the descriptioa which follows, and in part will be 20 
obvious from the description, or may be learned by practice 
of the invention. The objects and advantages of the invention 
may be realized and obtained by means of the instrum^- 
talities and combinations particularly pointed out in the 
appended claims. ^ 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accon:q>anying drawings, vf^ch are incorporated in 
and constitute a part of the spedfication, illustrate presently 
preferred embodiments of the invention, and together with 30 
the general description given above and the detailed descrip- 
tion of the preferred embodiments given below, serve to 
eiq)lain the principles of the inventioiL 

FIG. 1 is a perspective view showing a surgical manipu- 
lator system according to the first embodiment of the present 35 
invention; 

FIG. 2 is a block rfjagrflm of the control system for 
controlling the manipulators of the surgical manipulator 
system shown in FIG. 1; 

FIG. 3 is a perspective view of a man^ulator, for explain- ^ 
ing a first method of altering the coordinate system of each 
of the manipulators incorporated in the system shown in 

no. 1; 

FIG. 4 is a flow chart explaining the first method of 
altering the coordinate system of the manipulator; 

FIG. 5 is a flow chart explaining a modification of the 
method of altering the coordinate system of the manipulator 
shown in FIG. 3; 

FIG. 6 is a perspective view of a manipulator, for explain- 
ing a second method of altering the coordinate system of 
each of the manipulators incorporated in the system shown 
in FIG. 1; 

FIG. 7 is a flow chart explaiaing the second method of 
altering the coordinate system of the manipulator; 

FIG. 8 is a perspective view of a manipulator, for explain- 
ing a third method of altering the coordinate system of each 
of the manipulators incorporated in the system shown in 
FIG. 1; 

FIG. 9 is another perspective view of the same manipu- 
lator that is shown in FIG. 8, for explaining the method of 
altering the coordinate system of the man^ulator; 

FIG. 10 is a flow chart explaining the third method of 
altering the coordinate system of the manipulator; 

FIG. U is a schematic representation of the surgical- 65 
manipulator system according to the second embodiment of 
the invention; 
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FIG. 12 is a block diagram showing the controller used in 
the surgical manipulator system of FIG. U; 

FIG. 13 is a diagram explaining the operation of the 
surgical manipulator system shown in FIG. 11; 

FIG. 14 is a perfective view for explaining how the 
system of FIG. 11 is used to perform surgerjr, 

FIG. 15 is a perspective view showing the surgical 
manipulator system according to the third embodiment; 

FIG. 16 is a plan view of a console used as a command 
input device; 

FIG. 17 is a perspective view of a console having a 
joystick, used as a command input device; 

FIGS. 18A, 18B and 18C show three major components 
of the surgical manipulator system according to the fourth 
embodiment of the present inventioii; 

FIGS. 19Aand 19B are perspective views of the distal end 
portion of the manipulator incorporated in the fourth 
embodiment; 

RG. 20 is a perspective view of the manipulator control- 
ler incorporated in the surgical man^iulator system accord- 
ing to the fifth embodiment of the present invention; 

FIG. 21 is a perspective view of the manipolalor used in 
the fifth embodiment; 

FIGS. 22A and 22B are perspective views showing some 
of the components of the surgical nian^mlatof^ system 
according to the sixth embodiment of the invoition; 

FIG. 23 is a perspective view showii^ thi two manipu- 
lators iiKX)rporated in the seventh cmbodimmt of this inven- 
tiozs; 

FIG. 24 is a front view of the TV monitor used in the, 
seventh embodiment of the inyention; 

FIGS. 25A, 25B and 25C are perspective views showing 
the surgical manipulator system according to the eighth 
embodiment of the invention; 

FIG. 26 is a perspective view showing the surgical 
manipulator system according ^to the ninth embodiment of 
this invention; 

FIG. 27 is a perspective view, explaining how the surgical 
manipulator system according to the tenth embodiment of 
the invention is operated to perform a transcutaneous sur- 
gery; 

FIG. 28 is a block diagram illustrating the surgical 
manipulator system according to the eleventh embodiment 
of the present invention; 

FIG. 29A is a diagram schematically showing the head- 
mount display used in the eleventh embodiment; 

FIG. 29B is a diagram schematically showing the 3D 
scope used in the eleventh raibodiment; 

FIGS. 30A aiKl 30B show two types of sensors for 
detecting the orientation of the optical axis of the eye, which 
can be used in the eleventh embodiment; 

FIG. 31A is a perspective view of the master arm incor- 
porated in the eleventh embodiment; 

FIG. 31B is a perspective view of the slave manipulator 
used in the eleventh embodiment; 

FIG. 32 is a diagram explaining how to operate two 
master arms; 

FIG. 33 is a diagram for explaining how the slave 
manipulator incorporated in the eleventh embodiment is 
operated to perform a transcutaneous endoscope surgery; 

FIGS. 34A, 348 and 34C are flow charts explaining how 
the controller controls the slave manipulator in the eleventh 
embodiment; 
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FIG. 35 is a block diagram illustratiDg the surgical FIG. 57 is a side view of the bending mechanism of the 

manipulator system according to the twelfth embodiment of medical instrument shown in FIG. 56; 

this invention; FIGS. SSA and 58B are sectional views illustrating the 

FIGS. 36Ato 36C are diagrams explaining the operation free-lock mechanism for the medical instnmient shown in 

of the twelfth embodiment; S piG. 56; 

FIGS. 37A and 37B are perspective views of the eodo- FIG. 59 is a perspective view of the 3D scope and holder 

scope and the surgical manipulator, both incorporated in the section of the surgical manipulator shown in FIG. 50; 

thirteenth embodiment of the present invention; pjQ go is a side view of the fastaiing device used in the 

FIG. 38 is a diagram showing the entire surgical manipu- twentieth embodiment of the invention, for fastening a 

lator system according to the thirteenth embodiment; surgical manipulator to the operating table; 

FIG. 39 is a perspective view showing the surgical piGS. 61A and 61B are vertical sectional views of the 

manipulator system according to the fourteenth embodiment fastening device shown in FIG. 60; 

of the invention, and explaining how the system is used to pjQ shows the mechanisms supporting the manipu- 

implant an artificial blood vessel in the aorta; ^^^^ incorporated in the surgical manipulator system 

FIG. 40A is a longitudinal sectional view of an artificial according to the twenty-first embodiment of the present 

btood vessel and the endoscope used in the fourteenth invention; 

embodiment of the invration, the latter inserted in the FIG. 63 is a sectional view taken along line X—X in FIG. 

former; ^2; 

FIG. 40B is a front view of the endoscope incorporated in 20 ^ ^ perspective view of the surgical manipulator 

the fourteenth embodiment; ^ surgical mampulator system according to the 

FIG. 41 is a longitudinal sectional view of the aorta and twenty-second embodiment of this invention; 

the artificial blood vessel, the latter implanted in the former, piG. 65 is a perspective view of the master manipulator 

FIG. 42 is a schematic representation of the surgical ^idi is incorporated in the system according to the twenty- 

manq)ulator system according to the fifteenth embodiment 23 ^^^jj embodiment and wliich has an arm for generating an 

of the invention; operation command to the surgical manipolator, 

FIG. 43 is a sectional view of the coupler used in the pjQs. 66A and 66B arc diagrams of the point-tocking 

fifteenth embodiment, said view taken along a horizontal me<dianism for positioning the surgical manq>ulator shown 

plane; in rG. 64; 

FIGS. 44A, 44B and 44C arc diagrams for explaining the pjQg 57^ 573 are sectional vkws lowing the 

operation of the fifteenth embodiment; mechanism for driving the insertion section of the stugical 

FIG. 45 is a flow chart explaining the operation of the manipulator shown in FIG. 64; 

fifteenth embodiment; pIG^ 68 is a diagram explaining how to apptoadi to the 

HG. 46 is a sectional front view of one of the couplers 35 kidney the insertion section of the surgical manipulator used 

incorporated in the surgical man^ulator system according to in the twenty-third embodiment of the invention; 

the sixteenth embodiment of the invention; FIGS. 69A to 69D arc diagrams for explaining how the 

FIG. 47 is a sectional plan view of one of the couplers end effecter of the surgical manq>ulator shown in FIG. 68 is 

incorporated in the surgical manq)ulator system according to operated in order to suture an artery; 

the seventeenth embodiment of the invention; 40 FIGS, 70Ato 70D arc diagr ams showing various types of 

FIG. 48 is a diagram schematically illustrating the surgi- 3D position sensors; and 

cal manipulator system according to the eighteenth cmbodi- pjQ ^ ^ diagram showing a gyroscope mechanism, 
ment of the invention; 

FIGS. 49Aand 49B arc diagrams explaining the operation DETAILED DESdUFTIGN OF THE 

of the eighteenth embodiment; ^5 PREFERRED EMBODIMENTS 

FIG. 50 is a diagram schematically showing the surgical Embodiments of the invention will be described below, 

manipulator system according to the nineteenth embodiment with reference to the accompanying drawings, 

of the present invention; FIGS. 1 to 10 show a surgical manipulator system accord- 

FIG. 51 is a perspective view showing the nineteenth ^ Hjc first embodiment of the invention. This system is 

embodiment and explaining the operation thereof; characterized in that a slave manipulator and a master 

FIG. 52 is side view of the first manipulator incorporated manipulator for remote-controlling the slave manipulator 

in the nineteenth embodiment, whidi is equipped with a can be set in any desired relation in terms of the direction in 

surgical instrument; which they arc moved. 

FIG. 53 is side view of the second manipulator incorpo- 55 pio. 1 is a schematic representation of the surgical 

rated in the nineteenth embodiment, which is equipped with manipulator system. As shown in FIG. 1, the system com- 

a 3D scope; prises an operating table 1, two slave manipulators 3 and 4, 

FIG. 54 is a sectional view showing a mechanism for two master manqjulatois 16 and 17, and a console 18. A 

connecting a medical instrument or a 3D scope to the holder patient 2 lies on the table 1, on his or her back, 

section of the surgical manipulator used in the nineteenth 50 The slave manipulators 3 and 4 arc multi-joint ones, each 

embodiment; comprising a plurality of arms 5 which are jointed end to 

FIG. 55 is a sectional view showing a mechanism for end. The slave manipulators 3 and 4 are secured to the left 

connecting the bolder section and arm section of the surgical and right sides of the operating table 1, respectively. More 

manipulator used in the nineteenth embodiment; precisely, the base 7 of the first slave manipulator 3 is fixed 

FIG. 56 is a perspective view showing the internal struc- 65 to the right side of the table 1, and the base 8 of the second 

tures of the medical instrument and the holder section of the slave manipulator 4 to the left side of the table 1. The slave 

surgical manipulator; manipulators 3 and 4 extend from the respective bedsides 
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toward that part of the patient 2 on which surgery will be or 
is being performed. 

The distal ends of the slave manipulators 3 and 4 bold 
medical instruments 9 and 10, respectively^ each of whidi is, 
for example, a forceps, a knife, a suture device, a syringe or s 
the like. Instead, the distal end of each slave manipulator 
may hold an observation device, such as an endoscope, an 
ultrasonic echo probe, a surgical microscope or the like. 
Both slave manipiilators 3 and 4 are operated sudi that their 
distal end portions are inserted into a body cavity through an jq 
opening incised in the abdominal walL Transcutaneous 
surgery or transcutaneous endoscope surgery can thereby be 
accomplished. 

An actuator 11 and an encoder 12 are provided at a joint 
between any two adjacent arms of either slave manipulator. 
When driven by a controller, the actuator 11 rotates the first 
of the two acy acent arms with respect to the second arm. The 
encoder 12 detects the angle by which the first arm has been 
rotated. 

The slave manipulators 3 and 4 have spedfic base coor- ^ 
dinate systems 13 and 14, respectively. As shown in FIG. 1, 
each base coordinate system is an orthogonal one, having an 
origin at the base (7 or 8) and three axes Xq, Yq and Zq, 
Either slave manipulator has a task coordinate system 15, as 
well. The task coordinate system can be altered whenever ^ 
necessary, as will be explained later. The controller for both 
slave manipulators 3 and 4 receives information from the 
controller for the master man^ulators 16 and 17. Then, it 
sets the positions and orientations of the slave manipulators 
3 and 4 with respect to the specific coordinate system in 
accordance with the information, and controls the slave 
manipulators 3 and 4. 

A surgeon stands at the right side of the operating table 1, 
hence close to the first slave manqmlator 3. Hie master 
num^)ulators 16 and 17 are installed at the right side of the 35 
bed 1, too. They are fastened to the cansde 18. More 
specifically, the first master manq)ulator 16 has its base 25 
secured to the coiisole 18, whereas the second master 
man^;)ulator 17 has its base 26 secured to the console 18. The 
console 18 is spaced away from the operating table 1. ^ 
Alternatively, the console 18 may be secured directly to the 
right side of the operating table 1. 

The console 18 can be moved to any position desired, so 
that the surgeon may have easy access to it. In most cases, 
the console 18 is placed on the floor of the operating room. 45 
It may be suspended from the ceiling, instead. In any case, 
the console 18 is located at one side of the table 1 to ^able 
the surgeon to operate both master manipulators 16 and 17. 

The slave manipulators 3 and 4 are multi-joint ones, each 
comprising a plurality of aims 5 which are jointed end to 50 
end. An actuator 21 and an encoder 22 are provided at a joint 
between any two adjacent arms of either master manipulator. 
When driven by a controller, the actuator 21 rotates the first 
of the two adjacent arms with respect to the second ann. The 
encoder 22 detects the angle by which the first arm has been 55 
rotated. 

The master manipulators 16 and 17 have specific base 
coordinate systems 23 and 24, respectively. As shown in 
FIG. 1, each base coordinate system is an orthogonal one, 
having an origin at the base (25 or 26) and three axes Xq, Yq 60 
and Zq. The master manipulators share a task coordinate 
system 27. The task coordinate system 27 can be altered 
whenever necessary, as will be explained later. The control- 
ler for both master manipulators 16 and 17 sets the positions 
and orientations of the master manipulators 16 and 17 with 65 
respect to the specific coordinate system, arid controls the 
master manipulators 16 and 17. 
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FIG. 2 is a block diagram showing the control system for 
controlling the slave manipulators 3 and 4 and the master 
manipulators 16 and 17. As shown in FIG. 2, the control 
system has a first controller 31 for controlling the slave 
manipulators 3 and 4 and a second controller 32 for con- 
trolling the master man^ulators. The first controller 31 
comprises an MPU (Micro Processor Unit) 33, an actuator 
drive circuit 35 and an interface circuit 37. Similarly, the 
second controller 32 comprises an MPU 34, an actuator 
drive circuit 36 and an interface circuit 38. 

The MPU 34 calculates the positions of the distal ends of 
the master manipulators 16 and 17, i.e,, TCPs (Tool Center 
Points), as coordinate data for the space defined by the base 
coordinate systems 23 and 24 or by the task coordinate 
system 27. More precisely, the MPU 34 generates the 
coordinate data by perfiimung geometrical vector synthesis 
of the rotation angles of the arms constituting either master 
manipulator, which have been detected by the encoders 22 
provided at the joints among the aims. The coordinate data 
is transmitted via the interface circuit 38 to the interface 
circuit 37 of the first controller 31, and hence to the MPU 33 
thereof. 

The MPU 33 uses the coordinate data supplied from the 
MPU 34 of the second controller 32 and generates an 
operation command to the slave manq)ulatois 3 and 4. To be 
more ^cific, the MPU 33 calculates the positions of the 
distal ends of the slave marupulators 3 and 4, i.e., TCPs, as 
coordinate data for the ^ace defined by the base coordinate 
systems 13 and 14 or by the task coordinate system 15, and 
finds a deviation of this coordinate data from the coordinate 
data sullied from the MPU 34. The deviation, thus found, 
is the operation command. The operation command is sup- 
plied to the actuator drive circuit 35. The circuit 35 drives 
the slave manipulators 3 and 4 in accordance with the 
operation oommand. 

The deviation calculated by the MPU 33 of the first 
controller 31 is transmitted via the interface circuit 37 to the 
interface circuit 38 of the second controller 32, and hence to 
the MPU 34 thereof. The MPU 34 generates a force com- 
mand frx>m the deviation. The force command is supplied to 
the actuator drive circuit 36. The circuit 36 drives the 
actuators 21 on both master manipulators 16 and 17. The 
actuators 21 on each master manipulator generate a force 
whidi acts against the force the surgeon is applying on the 
master manipulator. The surgeon feels the force generated 
by the actuators 21. 

Referring bade to FIG. 1, as the surgeon operates the 
master manipulator 16, the slave inan^)ulator 3 which is 
associated with the master manipulator 16 is operated in the 
same way. In operation, the slave manipulator 3 follows the 
master manipulator 16. 

In the first embodiment, the first controller 31 controls the 
slave manipulator 3 such that the task coordinate system 15 
(no. 1) is identical in orientation to, but different in position 
from, the base coordinate system 23 of the master manipu- 
lator 16. The TCP of the master manipulator 16, or a point 
P present in the base coordinate system 23, corresponds to 
a point p present in the task coordinate system 15 of the slave 
manipulator 3. Thus, when the master manipulator 16 is 
moved, the slave manipulator 3 is moved in the same 
direction as the master manipulator 16. 

As shown in FIG. 1, the task coordinate system 27 of the 
master manipulator 17 is set, as described later, such that it 
is identical in orientation to, but different in position from, 
the base coordinate system 14 of the slave manipulator 4. 
The TCP of the master manipulator 17, or a point Q present 
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in the base coordinate system 27, corre^>onds to a point q 
present in the task coordinate system 14 of the slave manipu- 
lator 4. Therefore, when the master manipulator 17 is 
movedf the slave manipulator 4 is moved in the same 
direction as the master manipulator 17. 

The task coordinate system 27 common to the master 
manipulators 16 and 17 and the task coordinate system 15 
common to the slave manipulators 3 and 4 can be altered in 
various methods, each using vectors or matrices. Of these 
methods, the following three will be desc^bed. 
First Method 

This is called "three-point, touch-up method," which will 
be explained with reference to FIGS. 3 aixi 4. FIG. 3 is a 
perspective view of a manipulator 40 which is any one of the 
four manipulators 3, 4, 16 and 17 of the surgical manipulator 
system shown in FIG. 1 . FIG. 4 is a flow chart explaining the 
three-point, touch-up method. 

The position data of the TCP of the manq)ulator 40, which 
exists in a base coordinate syst^ 41, is obtained from the 
lengths of the arms 43 constituting the manipulator 40 and 
the rotation angles detected by the encoders 44 provided at 
the joints among the arms 43. 

To set, in a space, a new task coordinate system 45 
desirable for operating the manipulator 40 to perform 
suigery» an origin PI is first set for the task coordinate 
system 45. Then, the manipulator 40 is operated, moving the 
TCP to the origin PI. The controllers 31 and 32 read and 
store the position data of the origin PI, so that they can use 
the position data to define the position of the origin PI of the 
task coordinate system 45. 

Next, a provisional X axis is orientated in a direction 
sidtable for operating the manipulator 40 to perform surgery. 
This done, the manipulator 40 is operated, moving the TCP 
from the origin PI to a point P2 on the positive part of the 
provisional X axis. The controllers 31 and 32 read and store 
the position data of the point P2. At the same time, the first 
vector (PiPj)* extending from the origin PI to the point P2, 
is obtained, thus defining the X axis of the task coordinate 
system 45. (Here and hereinafter, no arrow is shown over the 
symbols of two points to indicate a vector.) 

Further, a point P3 whose Y-axis component is on the 
positive side is set in an XY plane desirable for operating the 
man^>ulator 40 to perform surgery. The manipulator 40 is 
then operated, moving the TCP to the point P3. The con- 
trollers 31 and 32 read and store the position data of the point 
P3. 

Then, the second vector (P1P3) extending from the origin 
PI to the point P3 is obtained. An outer product of the first 
and second vectors, i.e., {vector (PiP2)xvector (PiPj)}, is 
obtained. The direction of the outer product is the direction 
of the Z axis of the task coordinate system 45. Further, a 
product, {vector (PiP2)Kvector (PiP3)xvector (Pi?^}, is 
obtained. This product is the direction of the Y axis of the 
task coordinate system 45. 

As a result, the new task coordinate system 45 is defined 
in a space, by operating the manipulator 40, moving the TCP 
to three points in that space. 

The three-point, touch-up method may be modified. For 
example, a coordinate matrix M for the new task coordinate 
system 45 is calculated by applying the formulas (1), (2) and 
(3) presented in FIG. 4. To find the position vector Q of the 
TCP in the task coordinate system 45, the position vector P 
of the TCP in the base coordinate system 41, which is 
known, is multiplied, from the left, by a coordinate matrix 
M"\ Thus, Q-M"*P. FIG. 5 is a flow chart showing the 
sequence of steps for setting the new task coordinate system 
45. 



Second Method 

The second method of altering a task coordinate system 
will be explained, with reference to FIGS. 6 and 7. FIG. 6 
is a perspective view of a manipulator 40 which is any one 
of the four manq)ulators 3, 4, 16 and 17 of the system shown 
in FIG. 1. FIG. 7 is a flow chart explaining the second 
method. 

A matrix M for switching the base coordinate system of 
the manipulator 40 and the new task coordinate system 54 to 
be set in a space is expressed as: 
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where p(p^p ,pj, o(o^y,oJ, a(a^ay,aj, and n(n^,nj are 
the vector, close vector, directive vector and normd vector 
of the origin of the coordinate system, respectively. 

These variables can easily be calculated from the relation 
between the actual position of the manq>ulator 40 and the 
position the manipulator 40 assumes in the task coordinate 
system 45. More spedfically, the components n, o. a and p 
of the coordinate-switching matrix M v e set in a register of 
the MPU incorporated in the controller for the manipulator 
40. Then, the coordinate-switching matrix M is obttdned in 
accordance with the formula (A). Then, the position vector 
Q of the TCP in the task coordinate system 45 is obtained by 
multiplying the position vector P of the TCP in the base 
coordinate system 41, wfaidi is known, from the left, by a 
coordinate matrix M"^ Thus, Q-M"*P. 
Hiird Method 

The third method of altering a task coordinate system will 
be explained, with reference to FIGS. 8, 9 and 10. FIGS. 8 
and 9 are perspective views of a manipulator 40 which is any 
one of the four manipulators 3, 4, 16 and 17 of the system 
shown in FIG. 1. FIG. 10 is a flow (^art explaining the third 
method. 

First, there is defined a tool coordinate system 47. The 
tool coordinate system 47 is ^ledfic to the tool 46 mouinted 
on the manipulator 40; its origin is always the TCP of the 
manipulator 40. This coordinate system 47 is orientated in a 
direction suitable for operating the manipulator 40 to per- 
form surgery. Assuming that the tool 46 is an endoscope, the 
X axis of the tool coordinate system 47 is the axis of the 
endoscope, the Z axis of the system 47 extends in the offiset 
direction of the endoscope, and the Y axis of the system 47 
extend at right angles to the axis and the offset direction of 
the endoscope, respectively. To move the endoscope along 
its axis, it suffices to move the manipulator 40 along the x 
axis of the tool coordinate system 47. 

The tool coordinate system 47 is represented by H T 
(>M), which is a mathematical notation of the matrix M for 
switching the base coordinate system 41. The notation of H 
T should be written, by rule, in bold type, but is printed in 
ordinary type here and hereinafter. T is a matrix for switch- 
ing the coordinate system of the mechanical interface 
secured to the end cffccter of the manipulator 40. H is a 
matrix for switching the tool coordinate system 47. The 
matrix T can be geometrically obtained from the lengths of 
the arms 43 constituting the manipulator 40 and the rotation 
angles detected by the encoders 44 provided at the joints 
among the arms 43. The matrix H is expressed by a formula 
similar to the formula (A). The variables for the components 
of the matrix H may be calculated from the shape of the tool 
used, and the orientation of the tool coordinate system 47. In 
order to find the position vector Q the TCP takes in the base 
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coordinate system 14 which is known, it suffices to multiply 
the position vector P of the TCP in the base coordinate 
system 41, which is known, from the left, by a coordinate 
matrix M*^. 

The tool coordinate system 47 defined at a certain time is 5 
used as a task coordinate system to accomplish master-slave 
manipulation. For instance, the tool coordinate system 47 
defined when the TCP of the manipulator 40 is located at a 
point PI in a space is used as a task coordinate system. The 
tool coordinate system 47 is stored as the task coordinate lO 
system even after the man^)ulator 40 is operated, having its 
TCP shifted to a point P2 as shown in FIG. 8. 

Next, a matrix Ks for switching the coordinate system of 
a slave manipulator is defined, and a matrix Km for switch- 
ing the coorctinate system of a master manipulator is defined, 15 
too. The slave manipulator is operated such that the 
coordinate-switching matrix Ks becomes identical to the 
coordinate-switching matrix Km. As a result, the master 
manipulator and the slave manipulator are moved, keeping 
their positional relationship unchanged since tbeir task coor- 20 
dinate systems have been defined 

In oth^ words, the slave manipulator moves in the same 
direction as the master manipulator. This is an advantage. 
Unless the slave manipulator is operated such that the 
coordinate-switdung matrix Ks becomes identical to the 25 
coordinate-switdung matrix Km, it is difScult to maintain 
the master manipulator in a specific positional relation with 
the slave manipulator — particularly in a so-called unilateral 
master-slave manipulator system. In the unilateral master- 
slave manipulator system, the master manipulator has 30 
encoders but no actuators at its joints, and the master 
manipulator often deviates from the desired positional rela- 
tion with the slave manipulator. Once it has deviated &om 
that positional relation, the master man4)ulator will move in 
a direction the surgeon cannot expect at all. 35 

The second embodimrait of the present invention will now 
be described, with reference to FIGS. 11 to 14. The second 
embodiment is characterized in that a manipulator is moved 
in the same direction as a sensor (i.e., a remote-control 
means) is moved. 40 

FIG. 11 is a sdiematic representation of the surgical 
man^ulator system. As shown in FIG. 11, the system 
comprises a surgical manipulator 51 and a 3D position 
sensor 52. The manipulator 51 has a structure similar to the 
slave manipulators 3 and 4 incorporated in the first cmbodi- 45 
ment (FIG. 1). The manipulator 51 is secured to one side of 
an opiating table and is controlled in the same way as either 
slave manipulator used in the first embodiment. 

The 3D position sensor 52 is one designed to remote- 
control a manipulator, and is used to remote-control the 50 
surgical manipulator 51. To stale more precisely, the sensor 
52 comprises a source coil 53 and a sense coil 54. Either coil 
is substantially a cube and has three coil elements wound 
around three orthogonal axes, for generating and receiving 
magnetic fields. A drive circuit (not shown) supplies pulse 55 
currents, one after another, to the coil elements of the source 
coil 53. The three coil elements of the source coil 53 
generate three reference magnetic fields in the space occu- 
pied by the surgical manipulator system, as shown io RG. 
13, Hence, the coil elements constitute a transmitting sec- 60 
tioa The sense coil 54 detects the reference magnetic fields 
generated by the source coil 53 and determines its own 
position and orientation. Hence, the sense coil 54 functions 
as a receiving section. 

Both coils 53 and 54, or at least the sense coil 54 is small 6S 
enough to be gripped with the hand 56 as illustrated in FIG. 
11. /Jternatively, at least the sense coil 54 is small enough 
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to be niounted on a part of a body, for example, the head. The 
sense coil 54 may be fixed to the HMD (Head Mounted 
Display) which the surgeon wears, so that the slave man^ju- 
lator may be operated as the surgeon's head moves. 

As indicated above, the 3D position sensor 52 comprises 
the source coil 53 operating as a section for transmitting 
magnetic fields and the sense coil 54 functioning as a section 
for receiving the magnetic fields generated by the source soil 
53. As shown in FIG. 12, the sensor 52 further comprises a 
relative position detecting circuit 58 for detecting the por- 
tion and orientation of the sense coil 54, both relative to the 
source coil 53. Provided outside the sensor 52 is a manipu- 
lator controller 59. The controller 59 is designed to remote- 
control the surgical manqmlator 51 in accordance with the 
relative 3D position detected by the coils 53 and 54 of the 
3D position sensor 52. The 3D position sensor 52 and the 
manipulator controUer 59 constitute a remote-control system 
for controlling the manipulator 51. The 3D position sensor 
52 and the manipulator controller 59 constitute a remote- 
control system 55. 

When the sense coil 54 receives the reference magnetic 
fields emanating from the source coil 53, currents are 
induced in the coil elements of the sense coil 54, flowing 
along the axes of the coil elements, as is illustrated in FIG. 
13. The relative position detecting drciut 58 detects the 
vectors of the magnetic fields and calculates the relative 3D 
positioiis of the coils S3 and 54 firom the vectors of the 
magnetic fields. 

The 3D position sensor 52 transmits data representing the 
relative 3D positions of the coils 53 and 54 to the manipu- 
lator controller 59. The controller 59 remote-controls the 
manipulator 51 in accordance with the 3D poation of the 
sense coil 54 relative to the source coil S3, which has been 
detected by the 3D position sensor 52. 

The remote-control system 55 can be q>plied to the first 
embodiment, Lc., the surgical manipulator system shown in 
PIG. 1 — as illustrated in FIG. 14. As shown in FIG. 14, two 
slave manipulators 3 and 4 are secured at their bases 7 and 
8 to the ri^t and left sides of an operating table 1, respec- 
tively. IWo source coils 53, each fimctioning as a transmit- 
ting section, are secured to the right side of the operating 
table 1. Two sense coils 54, each functioning as a receiving 
section, are held in the hands of a surgeon or attadied to the 
head band the surgeon wears. The first source coil 53 and the 
first sense coil 54 constitute a first 3D position sensor 52a. 
The second source coil S3 and the second sense coil 54 
constitute a second 3D position sensor 526. 

The first slave manipulator 3 is operated in accordance 
with the relative 3D positions of the coils 53 and 54 of the 
first position detector 52a. Atask coordinate system 62 is set, 
in addition to the base coordinate system 61 of the first 3D 
position sensor S2a. The slave manipulator 3 is operated 
such that the vector P representing the position wtiich the 
sensor coordinate system 63 set for the sense coil 54 and 
assimies in the task coordinate system 62 coincides with the 
vector p representing the position which the TCP of the first 
slave manipulator 3 takes in the base coordinate system 13 
set for the first slave manipulator 3. The task coordinate 
system 62 can be freely altered, by using the sensor coor- 
dinate system 63 (an orthogonal coordinate system) set for 
the sense coil 54. For example, a sensor coordinate system 
63 set at a time may be used as the task coordinate system 
62. 

Once the task coordinate system 62 has been set at any 
desired position, the base coordinate system 13 of the slave 
manipulator 3 (in which the TCP is present) becomes 
identical in orientation to the task coordinate system 62 (in 



04/21/2004, EAST Version: 1.4.1 



5,876,325 



IS 



16 



10 



15 



20 



25 



30 



which the sensor coordinate system 63 exist). Hiis makes it 
easy for the surgeon to operate the slave manipulator 3. 

The second slave manipulator 4 is operated in accordance 
with the relative 3D positions of the coib 53 and 54 of the 
second position detector S2b. The second slave manipulator 
4 is operated such that the vector Q representing the position 
wiiich the sensor coordinate system 63 sets for the sense coil 
54 and assumes in the base coordinate system 61 set for the 
second position detector S2b coincides with the vector q 
representing the position which the TCP of the second slave 
manipulator 4 takes in the task coordinate system 65 set 
freely for the second slave manq^ulator 4. The task coordi- 
nate system 65 can be freely altered, by the first or second 
method applied in the first embodiment Once the task 
coordinate system 65 has been set, the vector Q and the 
vector q become identical in orientAtion as shown in FIG. 14. 
The surgeon can therefore operate the second slave manipu- 
lator 4 with ease. 

The third embodiment of the invention will be described 
with reference to FIG. 15. The third embodiment is charac- 
terized in that the manipulators are operated in accordance 
with audio commands made by a surgeon. 

FIG. 15 is a pcr^)cctive view showing the surgical 
manipulator system according to the third embodiment. As 
shown in FIG. 15, the surgical manipulator system com- 
poses a speech input device 90, and a speech recognition 
device 91, two slave manipulators 3 and 4. The system 
further comprises a manipulator controller, which is not 
shown. 

When a surgeon 92 utters an audio oonomand, a word such 
as "forward," "backward," "upward," "downward," "left- 
ward" or "rightward," to the speech input device 90 (e.g., a 
microphone), the speech recognition device 91 analyses the 
audio command and generates command data. The com- 
mand data is supplied to the manipulator controller (not 35 
shown). 

The slave manipulators 3 and 4 are secured at their bases 
7 and 8 to the ri^t and left sides of an operating table 1. For 
each slave man^tulator there is set a base coordinate system 
93 which has an up-down axis, a left-^ight axis and a 
forward-backward ads. Jhs base coordinate systems can be 
altered by any one of the methods described above, such that 
they are orientated to the direction in which the surgeon 92 
looks at the TCPs of the slave manq)ulator5 3 and 4. 

Usually, the base coordinate system 93 preset for the first 
slave manipulator 3 is orioitated in such a manner that the 
three axes indicate the up-down, left-right and forward- 
backward directions — all exactly the same as the three 
directions the suigeon 92 perceives with respect to the TCP 
of the manipulator 3. It is therefore unnecessary to alter the 
base coordinate system 93 of the first slave manipulator 3. 

On the other hand, the base coordinate system 93 preset 
for the second slave manipulator 4 is orientated such that the 
left-right axis and the forward-back axis indicate the left- 
right and forward-backward directions opposite to those the 
surgeon 92 perceives with respect to the TCP of the manipu- 
lator 4, though the up-down axis indicates the same up-down 
direction as the surgeon 92 perceives with respect to that 
TCP. This is inevitably because the second slave manipula- 
tor 4 is secured to the left side of the operating table 1 and 
located opposite to the first slave manipulator 3. 
Consequently, the audio commands of "leftward," 
"rightward," "forward" and "backward" made by the sur- 
geon 92 cannot be applied to the operation of the second 
slave manipulator 4. 

In order to operate the second slave manipulator 4 cor- 
rectly in accordance with any audio command the surgeon 
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92 issues, the base coordinate system 93 of the manipiilator 
4 is altered to orient the left-right and the forward-backward 
axes as indicated by broken-line arrows in FIG. 15. To 
accomplish this alteration it suffices to rotate the horizontal 
plane of the base coordinate system 93 by 180°. Once the 
base coordinate system 93 of the manipulator 4 has been so 
altered, it is quite easy for the surgeon 92 to operate the 
second slave manipulator 4, as well as the first slave 
manipulator 3. 

In the third embodiment, the slave man^ulators 3 and 4 
are moved in various directions in the base coordinate 
systems 93 in accordance with the corresponding audio 
commands the surgeon 92 makes. Instead, commands may 
be generated by operating a console 96 shown in FIG. 16, 
whidi has six switches 94, i.e., an "upward" switch, a 
"downward*' switch, a "leftward** switch, a **rightward" 
switch, a "forward" switdi and a "backward" switch. 
Alternatively, commands may be generated by operating a 
console 97 shown in FIG. 17, which has a joystick 96. 

The fourth embodiment of this invention wiH be described 
with reference to FIGS. 18A, 18B and 18C and FIGS. 19A 
andl9B. 

The fourth embodiment conq)rises a manqralator 100 
shown in FIG. ISA The manipulator 100 has an insertion 
section 101 which is to be inserted into a body cavity. The 
insertion section 101 has an observation device 102 in its 
distal end The observation device 102 is used to generate 
signals representing an image of the interior of the body 
cavity. The device 102 is, for example, a 3D scope IHQ 
shown in FIG. 19A or an ultrasonic probe 105 shown in FIG. 
19B which contains an ultrasonic transducer 104. 

The fourth embodiment further comprises a joystick 106 
and a TV monitor (display) 107 which are shown in FIGS. 
18C and 18B, respectively. The joystick 106 is used to 
remote-control the man^ulator 100. The TV monitor 107 is 
provided to display the image obtained by the observation 
device 102. 

A fixed tool coordinate system 108 is set at the distal end 
of the observation device 102. In this tool coordinate system 
108, the distal end of the insertion section 101 of the 
manipulator 100 is moved as the joystick 106 is operated 
along the X axis, Y axis and Z axis. The tool coordinate 
system set for the joystick 106 can be altered by any one of 
the methods described above. 

The tool coordinate systems of the joystick 106 and the 
TV monitor 107 are adjusted to become identical in orien- 
tation. Hence, as the joystick 106 is operated in any 
direction^ thus moving the distal end of the insertion section 
101, an object displayed on the TV monitor 107 moves in the 
same direction. This enhances the operability of the surgical 
manipulator system. 

The fifth embodiment of the invention, which is a surgical 
manipulator system, will be described with reference to 
FIGS. 20 and 21. 

This surgical manipulator system comprises a manipula- 
tor HO, an HMD 115 and a source coil 118. As shown in 
FIG. 21, the manipulator HO has an insertion section 111 
and an observation device 112 contained in the distal end of 
the insertion section 111. Like its counterpart of the fourth 
embodiment, the observation device 112 is, for example, a 
3D scope 103 shown in FIG. 19Aor an ultrasonic probe 105 
shown in FIG. 19B which contains an ultrasonic transducer 
104, 

As shown in FIG. 20, the HMD 115 is worn by a surgeon 
116 who operates the surgical manipulator system. The 
HMD 115 comprises a head band and a sense coil 117 
attached to the head band. The position of the surgeon's head 
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is detected bom the positional reUtion which the seose coil 
117 assumes with the source coil 118. Thus, the seose coil 
117 and the source coil 118 constitute a device for detecting 
the motion of the head of the surgeon 116. In accordance 
with the motion of the head, the manipulator 110 is moved 5 
in the same manner. The coordinate system of the head can 
be altered to have the same orientation as the coordinate 
system of the manipulator 110, or vice versa, so that the 
surgeon 116 may perform successful surgery. 

A fixed tool coordinate system 119 is set for the obser- lO 
vation device 112 contained in the distal end of the insertion 
section lU. Sensor coordinate systems 121 and 122 are set 
for the sense coil 117 and the source coil 118, respectively. 
The 3D position relation between the sense coordinate 
systems 121 and 122 is calculated in the form of a matrix M 15 
for switching a sensor coordinate system, from the sensor 
coordinate system 121 to the sensor coordinate system 122. 

It is inconvenient if the maiupulator 110 is operated in the 
same way as the surgeon 116 moves his or her head, as in the 
case where the surgeon 116 puts on or takes off the HMD 20 
115 or where the manipulator 110 must be kept in the same 
position. To cope with such cases, the system may be 
equipped with a diangeover switdi for enablLig the manipu- 
lator 110 to move or prohibiting the manipulator 110 from 
moving, as the surgeon 116 moves his or her head, thus 25 
moving the HMD 115. 

Assume that the changeover switch has been operated, 
enabling the manipulator 110 to move as the head moves. In 
this case, the manqnilator 110 can be moved such that the 
matrix P for switching the sensor coordinate system 121 set so 
the moment the changeover switch has been so operated, to 
the present sensor coordinate system 121', coincides with the 
matrix Q for switching the tool coordinate system 119 set the 
moment the changeover switch has been so operated, to the 
present tool coordinate system 119'. No discrepancy there- 35 
fore exists between the directions in which the head and the 
HMD 115 move at the time of operating the diangeover 
switdi. 

The sixth embodiment of the present invention will be 
described with reference to FIGS. 22A and 22B. 40 

The sixth embodiment comprises a surgical man^ulator 
110 shown in FIG. 22A, an HMD 115 shown in FIG. 22B 
and a manipulator 127 shown in FIG. 22B. The surgical 
man^)ulator 110 is identical to the manipulator (FIG. 21) 
incorporated in the fifth embodiment. The HMD 115 is of the 45 
same type as the HMD (FIG. 20) used in the fifth embodi- 
ment; it is provided to display an image obtained by the 
observation device 112 wtiich is contained in the di&taJ end 
of the insertion section 111 of the manipulator 110. 

The manipulator 127 is secured at an upper end to the 50 
ceiling of an operating room. The HMD 115 is connected to 
the lower end of the manipulator 127. The manipulator 127 
is a multi-joint one, comprising a plurality of arms 126 and 
encoders 128 provided at the joints among the anns 126. As 
the surgeon 116 moves his or her head, the arms 126 of the 55 
manipuialor 127 are rotated. The encoders 128 detect the 
rotation angles of the arms 126. The position the HMD 115 
takes in space is calculated from the rotation angles detected 
by the encoders 128. In accordance with the position of the 
HMD 115, the surgical manipulator 110 is operated in the 60 
same manner as the surgeon 116 moves his or her head. A 
fixed tool coordinate system 119 is set at the distal end of the 
observation device 112, and a fixed sensor coordinate sys- 
tem 121 is set for the HMD 115. 

It is inconvenient if the manipulator HO is operated in the 65 
same way as the surgeon 116 moves his or her head, as in the 
case where the surgeon 116 puts on or takes off the HMD 
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115 or where the manipulator 110 must be kept in the same 
position. To cope with such cases, the sixth embodiment is 
equipped with a changeover switch for enabling the man^u- 
lator 110 to move or prohibiting the manipulator 110 from 
moving, as the surgeon 116 moves his or her head, thus 
operating the manipulator 127. 

Assume that the changeover switch has been operated, 
enabling the manipulator 110 to move as the head moves. In 
this case, the manipulator 110 can be moved such that the 
matrix P for switching the sensor coordinate system 121 set 
the moment the changeover switch has been so operated, to 
the present sensor coordinate system 121', coinddes with the 
matrix Q for switdiing the tool coordinate system 119 set the 
moment the changeover switch has been so operated, to the 
present tool coordinate system 119'. There is no discrepancy 
between the directions in wtiich the head and the HMD 115 
move at the time of operating the diangeover switch. 

The seventh embodiment of this invention will be 
described with reference to FIGS. 23 and 24. The seventh 
embodiment is characterized in that an observation manq>Q- 
lator 133 and a TV monitor 135 are used to enhance the 
operability of a surgical manipulator 130. 

As shown in FIG. 23, a tool coordinate system 131 is set 
at the distal end of the surgical manqiulator 130, such that its 
X axis aligns with the bngitudinal axis of tte surgical 
manipulator 130. An observatiozi device 134 is provided in 
the distal end of the observation manipulator 133. The 
device 134 wfaidi scans the surgical manipulator 130 system 
according to the seventh embodiment comprises a suigical 
manipulator 130 and generates a signal representing the 
image of the manipulator 130. The signals are supplied to the 
TV monitor 135 shown in FIG. 24, hereby the image of the 
surgical manipulator 130 is displayed on the screen of the 
TV monitor 135. The TV monitor 135 displays the tool 
coordinate system 131, too, superimposed on the image of 
the surgical manipulator 130. 

The seventh embodiment comprises a controller for mov- 
ing the suigical manipulator 130 along the three axes of the 
tool coordinate sy^m 131. As the manipulator 130 is 
moved, its image on the screen of the TV monitor 135 moves 
in the same way. Hence, a surgeon can operate the controller, 
thus moving the manipulator 130 in various manners, while 
looking at the image of the surgical manipulator 13 moving 
on the screen of the TV monitor 135. 

The eighth embodiment of the invention will be described 
with reference to FIGS. 25A, 25B and 25C. This embodi- 
ment is a surgical manipulator system having a slave 
manipulator 151 and a master manipulator 161. 

As shown in FIG. 25B, the slave manipulator 151 com- 
prises a surgical unit 153 and a robot 154. The \xmi 153 has 
an insertion section 152, the distal end portion of whidi is 
to be inserted into a body cavity. The robot 154 has a 
plurality of axes and can rotate the insertion section 152 and 
extends the section 151 back and forth. As illustrated in FIG. 
25C, the slave manipulator 151 further comprises a 3D 
scope 155 and a pair of instruments 156 and 157. The 3D 
scope 155 is provided in the distal end of the insertion 
section 152. Both instruments 156 and 157 extend through 
the insertion section 152 and protrude forward firom the 
distal end thereof. The instruments 156 and 157 can be bent 
in two or more degrees of freedom. 

The master manipulator 161 is a multi-joint one and 
serves as controller for remote-controlling the slave manipu- 
lator 151. It is secured at its upper end to the ceiling of an 
operating room. Connected to its lower end are a HMD 162 
and a pair of operation arms 163 and 164. The operation 
arms 163 and 164 are associated with the instruments 157 
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and 156 which protnide firom the distal end of the insertion instance^ the ninth embodiment may have only one master 

section 152. A coordinate system is set for either ann, so that manipulator 16 for lemote-controUing both slave man^>ii- 

the surgeon may operate the arm to remote-control the later 3 and 4. In this case, a slave-switching device is 

instrument (157 or 156) associated with the arm. operated to select the slave man^ulator 3 or the slave 

More specifically, five coordinate systems are set for the 5 manipulators 4 so that the selected slave manipulator may be 

3D scope 155, the instruments 156 and 157 and the opera- remote-controlled when the ma^er manq>Tilator 16 is opcr- 

tion arms 163 and 164, respectively. When the surgeon ated, 

operates the master manipulator 161 in the tool coordinate The tenth embodiment of this invention will be described 

system 165 set for the master manipulator 161, the insertion with reference to FIG. 27. This onbodiment comprises two 

section 152 of the slave manipulator 151 is operated in the lO slave manipulators 181 and 182 and an encbscope 183. A 

same way in the tool coordinate system 166 set for the slave surgeon inserts the manipulators 181 and 182 and the 

manipulator 161. The slave manipulator 151 is operated, endoscope 183 into a body cavity through an incision made 

such that the sensor coordinate system 167 set for the HMD in the body wall. He or she then performs a transcutaneous 

162 with respect to the tool coordinate system 165 corre- surgery, by operating two master manipulators (not shown) 

sponds to the scope coordinate system 170 set for the 3D 15 of the same type as used in the first embodiment (FIG. 1), 

scope 155 with respect to the tool coordinate system 166. thereby remote-controlling both slave manipulators 181 and 

The control coordinate systems 168 and 169 are set for the 182, while watching the interior of the body cavity through 

operation arms 163 and 164 with respect to the tool coor- the endoscope 183. 

dinate system 165, so that they correspond to the instrument In the first to tenth embodiments described above, the 

coordinate systems 171 and 172 set for the instruments 156 20 coordinate systems set for the suigical manipulators or the 

and 157 with respect to the tool coordinate system 166. coordinate system set for the means for remote-controlling 

Therefore, the 3D scope 155 and the instruments 156 and the surgical manipulators, or all these coordinate systems, 

157 can be operated in good coordination, though they arc can be altered to align in terms of orientatioa As a result, the 

controlled independently. surgical manq)ulators are moved in the same direction as a 

Tbe ninth embodiment of the present invention will be 25 surgeon moves the remote-contiol means, overcoming the 

described with reference to FIG. 26. The ninth embodiment restriction imposed in the case where the coordinate systems 

is a modification of the first embodiment. The ninth embodi- of the surgical manipulators differ in orientation firom the 

ment comprises two slave manipulators 3 and 4 and two coordinate system of the remote-control means. In other 

master manq>ulatois 16 and 17. The slave manq>ulators 3 words, the surgical manipulators can be operated in the same 

and 4 are secured to the right and left sides of an operating 30 maimer as the surgeon operates the remote-control means, 

table 1. The master manipulators 16 and 17, whidi remote- whichever positions the surgical manipulator take with 

control the slave manipulators 3 and 4, respectively, arc respect to the remote-control means, 

secured to the same side (Le., rigjil side) of the operating The eleventh embodiment of this invention will be 

table 1, not to a console as in the first embodiment (FIG. 1). described, with reference to FIG. 28, FIGS. 29A%d^ 29B, 

Hence, the first slave manipulator 3 and the first master 35 FIGS. 3QA and 30B, FIGS, 31A and 31B, HG. 32, FIG. 33 

manq)ulator 16, which are associated, are located at the and FIGS. 34A to 34C. 

same side of the operating table 1, whereas the second slave As shown in FIG. 28, the eleventh embodiment, or a 

manipulator 4 and the second master manipulator 17, which surgical manipulator system 230, comprises a slave manqiu- 

are associated, are located at the opposite sides of the lator 201, an HMD 208, a master arm 209 and a controller 

operating table 1. 40 207. 

Since the second slave manipulator 4 opposes the second The slave man^julator 201 is designed to be inserted into 

master manipulator 17. across the operating table 1, its base a body cavity to perform a physical action on an object of 

coordinate system differs in orientation firom the base coor- surgery which exists in the body cavity. The slave manipu- 

dinate system of the second master manipulator 17. lator 201 comprises a multi-joint arm 202, a gripper 203, a 

Cot^qucntly, when a surgeon moves the second master 45 toudi sensor 204, a drive section 205 and a 3D scope 206. 

manipulator 17, the second slave manipulator 4 is moved in As shown in FIG. 31B, the gripper 203 is atUched to the 

a different direction, as has been explained in conjunction distal end of the multi-joint arm 202. As shown in FIG. 28, 

with the first embodiment. the sensor 204 is atUdied to the gripper 203. The drive 

To operate the slave manipulators 3 and 4 in the same way section 205 is provided to drive the multi-joint arm 202, 
as the second master manipulators 16 and 17, respectively, 50 rotating the joints of the arm 202 by various angles so that 

one task coordinate system 173 is set for both slave manipu- the arm 202 is bent in a desired fashion. As is shown in FIG. 

lators 3 and 4, and one task coordinate system 174 is set for 31B, the 3D scope 206 is located near the multi-joint arm 

both master manipulators 16 and 17. This measure taken, the 202, for providing a 3D image of the interior of the body 

operability of the surgical manipulator system can be cavity. 

improved, no matter how many master-slave sets of manipu- 55 The controller 207 is provided to control the slave 

lators are provided and no matter where the slave manipu- manipulator 201 and to process the left-eye image signal and 

lators are arranged with respect to the associated master the right-eye image signal generated by the 3D scope 206. 

manipulators. Namely, the ninth embodiment solves the The HMD 208 is designed to receive the image signals 

problem which arises in the case where, as in most surgical processed by the controller 270 and display the left-eye 
manipulator systems, the master and slave manipulators arc 60 image and right-eye image of the interior of the body cavity, 

so arranged with respect to the patient that they caimot have The master arm 209 functions as remote-control means for 

identical base coordinate systems. causing the controller 207 to drive the slave manipulator 

As in the ninth embodiment, master manipulators may be 201. 

provided in the same number as the slave manipulators used. The master arm 209 is operated by a siirgecn as illusU:ated 
Instead, less master manipulators may be provided than 65 in HG. 31A. As the surgeon operates the master arm 209, the 

slave manipulators. If this is the case, one master manipu- controller 207 calculates the position which the multi-joint 

lator is associated with a plurality of slave manipulators. For arm 202 should take to perform surgery in the body cavity. 
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The suigical manipulator system 230 may have two or more manipulator 201, controlling both variable focal*poiat units 

slave manipulators, instead of only one, and the system 230 of the 3D scope 206. Namely, the circuit 215 adiieves 

has as many master arms. For example, in the case where the so-called automatic focusing. 

system 230 has two slave manipulators, the system 230 has The displacement calculating circuit 216 supplies the 

two master arms 209 as shown in FIG. 32. In this instance, s displacement signal to the control circuit 217. In accordance 

the surgeon 210 operates both master arms 209 as shown in with the deviation signal the circuit 217 controls the drive 

FIG. 32, while looking at a 3D image of the interior of a section 205 of the slave manipulator 201. Thus controlled, 

body cavity which is displayed by the HNfD 208 he or she the drive section 205 drives the multi-joint arm 202, 

wears. whereby the TCP of the gripper 203 is moved to the TCP 

As illustrated in FIG. 296, the 3D scope 206 has, in its lo whic± the surgeon 210 has indicated by operating the master 

distal end, two objective optical systems 209a and 2096 and aim 209. The control circuit 217 is designed to stop the slave 

two CCDs 210a and 2106. The optical systems 209a and manipulator 201 when the pressure which the gripper 203 

2096 are arranged, each having a cooverg^ce angle with applies to the object 232 and whidi the touch sensor 204 

respect to the object present in the body cavity. Hie first detects increases above a preset value. As soon as the slave 

CCD 210a receives a left-eye optical image whidi the first is manipulator 201 is stopped, the gripper 203 is opened to 

objective optical system 209a has formed, and converts the release the object 232, for safety. 

le^-eye optical image into a lefi-eye image signal The There are two types of sensors which can be used as the 

second CCD 2106 receives a right-eye optical image which sensors 213a and 2136. The first type, shown in RG. 30A, 

the second objective optical system 2096 has formed, and includes a CCD camera 218 for detennining the position of 

converts the light-eye optical image into a rig^t-eye image 20 the eye (212a or 2126) and the center of the pupil (221a or 

signal The two image signals are supplied to the controller 2216) and subsequeotly detecting the orientation of the 

207. optical axis of the eye from the positional relation between 

The objective optical systems 209a and 2096 have one the eye and the center of the pupiL The second type, shown 

variable fbcal-point unit (not ^own) each. The unit may be in FIG. 30B, has a light-emitting diode 219 and a PSD 

a combination lens unit wfaidi has one optical axis and 2S (Position Sensing Device) 220. The light-emitting diode 219 

which focuses an optical image onto a CCD by means of emits an infrared beam to the eye (212a or 212^ and the 

pupil division. The variable focal-point unit adjusts the focal PSD receives the beam reflected from the eye, to aetect the 

distance under the control of the controller 207. orientation of the optical axis of ttie eye^ Both types of 

As shown in FIG. 28, the controller 207 comprises a sensors are known. ^'Hiotogiaph Industry,'' Shashin Kogyo, 

signal processing circuit 211, an aridmietic processing unit 30 January 1993, pp. 63 and 64 and pp. 104 and 105 disdoses 

214 and a control circuit 217. a system wherein a microcomputer calculates the rotation ^ 
The signal processing circuit 211 receives the left-eye angle of the eye from the positional relation between the 

image signal and the right-eye image signal from the CCDs pupil and an image formed by an infrared beam reflected 

210a and 2106, and processes these signals into video from the cornea, thereby to determine the orioitation of the 

signals of standard type. The video signals are supplied to as optical axis of the eye. 

the HMD 208. As shown in FIG. 29 A, the HMD 208 has two While the surgeon 210 is observing the virtual image 232* 

display sections 208a aiKl 2086, whidi display the lefl-eye of the object 232, the optical axes of his or her eyes 212a and 

image and the right-eye image, respectively. The surgeon 212 intersect with each other at the virtual image 232', as is 

210 sees these images with his or her left eye 212a and 2126, illustrated in FIG. 29 A. Hence, the position of the virtual 

thereby perceiving a 3D image of the object 232 present in 40 image 232' can be determined from the orientations of the 

the body cavity. That is, the objective optical systems 209a optical axes which the sensors 213a and 2136 have detected, 

and 2096 receive the real image of the object 232 with The position the object 232 assumes with respect to the 3D 

paralUx as ^own in FIG. 29B, and the display sections 208a scope 206 is determined from the position of the virtual 

and 2086 of the HMD 208 display the left-eye and right-eye image 232' formed by the HMD 208, since the optical 

images of the object 232, respectively, with parallax. Seeing 45 features of the HMD 208 and those of the 3D scope 206 are 

these images with the left and right eyes, the surgeon 210 known. The optical features of the HMD 208 are the 

perceives a 3D virtual image 232' of the object 232. positional relation between the eye 212a and the display 

As illustrated in FIG. 29A, the HMD 208 has sensors section 208a and the positional relation between the eye 

213a and 2136 for detecting the orientations of the optical 2126 and the display section 2086. The optical features of 

axes of the surgeon's eyes 212fl and 2126, respectively. 50 the 3D scope 206 are the positional relation between the 

These sensors 213a and 2136 are connected to the arithmetic objective optical system 209a and the CCD 210a, the 

processing unit 214 of the controller 207. The arithmetic positional relation between the objective optical system 

processing unit 214 comprises a position calculating circuit 2096 and the CCD 2106, and the magnifications of the 

215 and a displacement calculating circuit 216, The circuit optical systems 209a and 2096. 

215 receives the signals output from the sensors 213a and 55 It is desirable that the eye 212a and the display section 

2136 and representing the orientations of the optical axes of 208a have a positional relation similar to the positional 

the eyes 212a and 2126, and calculates the position of the relation between the objective optical system 209a and the 

object 232 from these signals. The circuit 216 compares the CCD 210fl, and that the eye 2126 and the display section 

posilionof the object 232, calculated by the circuit 215, with 2086 have a positional relation similar to the positional 
the position thereof, indicated by the master arm 209, 60 relation between the objective optical system 2096 and the 

thereby finding a deviation between the position of the CCD 2106. Is so, then: 9L-4L and OR-^R, where 8L and 4>R 

object 232 and that of the TCP which the surgeon 210 has are angles at which the optical axes of the eyes 212a and 

indicated by operating the gripper 203. The circuit 215 2126 incline to the optical axes of the object optical systems 

generates a signal representing the deviation, 209a and 2096 when both optical systems 209fl and 2096 
Ihe position calculating circuit 215 generates a signal 65 have their focal points at the object 232, 6L is an angle 

representing the position of the object 232 it has detected, between the axis A of the cornea 221a of the left eye 212a 

This signal is supplied to the 3D scope 206 of the slave and the optical axis of the display section 208a, and GR is an 
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angle between the axis A of the comea 221^1 of the left eye 
2126 and the optical axis of the display section 2086. 

The operation of the controller 207 will now be explained 
with reference to FIG. 33 and FIGS. 34A, 34B and 34C 

FIG. 34A is a flow diart explaining how the orientations 5 
of the optical axes of the surgeon's eyes arc detected to 
determine the position an object assumes in a body cavity, 
and how the automatic focusing is performed in the 3D 
scope 206 in accordance with the position of the object. FIG. 
34B is a flow chart showing how the slave manipulator 201 lo 
is operated and moved toward the position of the object 
which has been determined tmm the orientations of the 
optical axes of the eyes. FIG. 34C is a flow chart explaining 
how the position of the object, determined from the orien- 
tations of the optical axes of the eyes, is applied to prevent 
the slave manipulator 201 from beiisg remote-controlled 
erroneously. In the remote-control method of FIG. 34C 
which difl^ers from the method of FIG. 34B, the slave 
manipulator 201 can be remotc-oontroUod as the surgeon 
210 operates the master arm 209 if the TCP of the slave 
man^ulator 201 is present within a picdctcnnined radius 
£tom the point which corresponds to the TCP of the master 
ama 209. 

FIG. 33 is a diagram illustrating how the suigeon 210 
remote-controls the slave manq)iilator 201, thereby perform- 
ing a transcutaneous endoscope surgery on a patient 231. As 
shown in FIG. 33, the distal end portbn of the slave 
manq)ulator 201 is inserted into the body cavity 233, guided 
through a trocar 222 fitted in an opening incised in the 
abdominal wall of the patioit 231. The 3D scope 206 30 
provided in the distal end of the slave manq>ulator 201 scans 
the interior of the body cavity 233 and generates a left-eye 
image signal and a right-eye image signaL The two image 
signals arc supplied to the controller 207, which processes 
the signals into video signals of standard type. The video 35 
signals are supplied to the HMD 208, y/\ach forms a 3D 
image. Looking at the 3D image, the surgeon 210 operates 
the master arm 209. Tlie distal end portion of the slave 
manq)ulator 201 is operated in the body cavity 233 — in the 
same way as the master arm 209. Surgery is thereby per- 40 
formed in the body cavity 233. 

With reference to the flow chart of FIG. 34A, it will be 
explained how the orientations of the optical axes of the 
surgeon's eyes are detected to determine the position an 
object assumes in a body cavity, and how the automatic 4S 
focusing is performed in the 3D scope 206 in accordance 
with the position of the object . 

First, in Step SI, the surgeon 210 recognizes a virtual 
image 232' displayed by the HMD 208. Then, be or she 



15 



20 



25 



scope 206 such that automatic focusing is accomplished in 
the 3D scope 206. 

For sin^lidty of explanation, how to detect the position 
of the object has been described, with reference to FIGS. 
29A and 29B and FIGS. 34A which show and describe as if 
the optical axes of the surgeon's eyes, the axes of the optical 
units incorporated in the 3D scope 208 and the HMD 208 
were present in a plane, not in a space. Nonetheless, it may 
be understood from the above that the position the object 
assumes in a space can be detected in the eleventh embodi- 
ment. 

With reference to the flow chart of FIG. 34B, a method of 
remote-controlling the slave manipulator 201 will be 
explained. When controlled by the controller 207, the drive 
section 205 drives the multi-joint arm 202, whereby the TCP 
of the gripper 203 is moved to a point Q wtdch is the position 
of the object determined by the position calculating circuit 
215. 

With r^erence to the flow chart of FIG. 34C, another 
method which the controller 207 effects to remote-control 
the slave manipulator 201 will be explained. 

In this method shown in FIG. 34C, it is determined in Step 
S8 whether or not |P-Q|<R, where P is the position of the 
TCP of the master arm 209 (FIG. 31A), Q is that position of 
the object which has been determined from the orientations 
of the optical axes of the surgeon's eyes, and R.is a radius 
within ^vbict the TCP of the griper 203 cant^ moved 
without damaging the object or the multi-joint arm 202, or 
both. In the case shown in FIG. 31B, the point Q coindides 
with the TCP of the snpp^T 203. 

The control circuit 217 can remote-control the drive 
circuit 205 of the slave manipulator 201 only while the 
suigeon 210 looks at a point which exists within a radius of 
R from the TCP of the multi-joint aim 202. Stated in andt)ier 
way, the multi-joint arm 202 can be driven by the control 
circuit to have its TCP move toward the point Q correspond- 
ing to the point P (i.e., the TCP of the master arm 209), 
provided that pP-Q|<R. , 

Assume that the TCP of the slave manipulator 201 is 
located at the point Q which corresponds to the point P 
where the TCP of the master arm 209 is located. When the 
surgeon 210 shifts his or her eyes to a point Ql shown in 
FIG. 31B, it is determined in Step 88 that |P-Q|<R, Tliat is, 
YES in Step S8. The flow goes to Step S9, in which the 
control circuit 217 can further control the drive section 205. 
When the surgeon 210 operates the master arm 209, moving 
the TCP of the arm 209 to the point P, the control circuit 217 
controls the drive section 205, which drives the multi-joint 
arm 202. The TCP of the slave manipulator 201 is thereby 



looks at the virtual image 232' in Step S2. In Step S3, the 50 moved to the point Q which corresponds to the point P. On 



sensors 213a and 2136 incorporated in the HMD 208 detect 
the angles OL and 6R, both defined and described above, 
thus detectirig the orientations of the optical axes of the 
surgeon's eyes 212a and 2126, respectively. From the ori- 
entations of the optical axes of the eyes, the position 
calculating circuit 215 calculates the position of the position 
of the object. More precisely, in Step S4, the circuit 215 
compensates for the. geometrical diffcrerxx between the 
optical system of the 3D scope 206 and that of the HMD 
208, in accordance with the angles 6L and 6R, both defined 
and described above. The angles and ^R, at wtiich the 
optical axes of the eyes 212a and 2126 incline to the optical 
axes of the object optical systems 209a and 2096, are 
thereby obtained in Step S4, Based on the (t>L and (jiR, the 
position of the object is determined in Step S5 by the same 65 
method as employed in trigonometrical survey. Then, in 
Step S6, the position calculating circuit 215 controls the 3D 
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the other hand, when the suigeon 210 shifts the eyes to a 
point Q2 shown in FIG. 316, it is determined in Step S8 that 
tP-Q|>R, Namely, NO in Step S8. The flow goes to Step 
SIO, in whidi the multi-joint arm 202 can no longer be 
driven. 

Thus, in the eleventh embodiment, unless the surgeon 210 
looks at a point present within a radius of R from the TCP. 
of the slave manipulator 201, the slave manipulator 201 
cannot be operated even if an operation command is issued 
from the master arm 209. Neither the object nor the multi- 
joint arm 202 is damaged. This helps increase the safety of 
transcutaneoiis endoscope surgery. 

When tired, the surgeon 210 can hardly keep his or her 
attention to the object. His or her attention may Leave the 
object, though momentarily, and his or her eyes shift from 
the virtual image of the object If this happens, the slave 
manipulator 201 can no longer be operated, and the object 
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and the multi-joint arm can therefore be protected from coil 226a has acquired by detecting the intensity of the 

damage. In view of this, the eleventh embodiment is advan- magnetic field emanating from the so\irce coil 2266 and 

tageous. which represents the six degrees of freedom pertaining to the 

Thctwclfthcmbodimentof the present invention will now position and inclination of the HMD 208A. From this 

be described, with reference to FIG. 35 and FIGS. 36A, 36B s information the control circuit 227 determines the TCP of 

and 36C. The components of this embodiment which are the observation manipulator 225. Further, the control dicuit 

identical to those of the eleventh embodiment, are desig- 227 controls the surgical manipTilators 224a and 224b as the 

nated by the same reference numerals in FIGS. 35-36C, and surgeon 210 operates the master manipulators 228a and 

will now be described in detail. 2286, provided that the position of the object, which has 

As shown in FIG. 35, the twelfth embodiment, or a lo been obtained from the orientations of the optical axes of the 

surgical manqiulator system 235, comprises a controller surgeon's eyes 212a and 2126 by the sensors 213a and 2136 

207, a HMD 208A, two surgical manipulators 224a and of the HMD 208Aby the same method used in the eleventh 

2246, an observation manipulator 225, and two master embodiment, is within a predetermined distance from the 

manipulators 228a and 2286. The controller 207 remote- TCPs of the surgical manq}ulators 224a and 2246. 

controls the surgical man^ulators 224a and 2246 in accor- 15 With the twelfth mbodiment it is possible for the surgeon 

dance with the operation of the master manipulators 228a 210 to remote-control both surgical manq>ulatois 224a and 

and 2286. The controller 207 controls the observation 2246, while looking at the 3D virtual image of the object 

manipulator 225, as well, in accordance with the motion of located in a body cavity and maintaining the (d>servation 

the HMD 208A manipulator 225 at a desired position in the body cavity. The 

As shown in FIG. 36A, the distal end portions of the lo surgeon 210 therefore has a good sense of immediacy. Like 

surgical manq)ulators 224a and 2246, and the distal end the eleventh embodiment, the twelfth embodiment enables 

portion of the inserted on manipulator 225 are inserted into the surgeon 210 to perform a transcutaneous endoscope 

a body cavity, so that surgery may be performed on an object surgery in high safety. 

which exists in the body cavity. A 3D scope 225a is provided As described above, in the eleventh and twelfth embodi- 

in the distal end of the observation manipulator 225. 25 ments of the invention, any surgical manipulator is not 

As shown in FIG. 35, the controller 207 comprises a operated when the surgeon's eyes shift too far from the 

signal processing circuit 211, an arithmetic processing unit virtual image of the object, even if an operation command is 

214 and a control circuit 227. The arithmetic processing unit issued from a master manipttlator. No excessive force is 

214 comprises a position calculating circuit 215 and a therefore exerted on the object or any surgical manipulator 

displacement calculating circuit 216A. 30 even if the surgeon unintentionally operates a master 

The HMD 208A is identical to the HMD 208 shown in manipulator in an erroneous way. In view of this, both the 

FIG. 28, except that it has an additional component, i.e., a eleventh embodiment and the twelfth embodiment enable 

position sensor 226 for acquiring information on six degrees the surgeon 210 to perform a transcutaneous endoscope 

of freedom which pertain to the position and inclination of surgery in safety. 

the HMD 208A. The position sensor 226 comprises a sense 35 The thirteenth embodimwt of the invention will be 

coil 226a and a source coil 2266. As shown in FIG. 36B, the des(^*bed with reference to FIGS. 37A and 37B and FIG. 38. 

sense coil 226a is wound around three axes wfaicdi extend at As shown in FIG. 38, the thirteenth embodiment com- 

right angles to one another. The source coil 2266 is fixed prises an endoscope 301 and a surgical manipulator 302. The 

steadfastly. The sense coil 226a detects the intensity of the endoscope 301 has an insertion section 304 which can be 

magnetic field emanating frt>m the source coil 2266, thereby 40 inserted into the abdominal cavity. The surgical manipulator 

acquiring the information on the six degrees of freedom 302 has an insertion section 308 which can be inserted into 

pertaining to the position and inclination of the HMD 208A. the cavity 307. In the instance shown in FIG. 38, the 

The displacement of the HMD 208A, detected by the insertion section 304 of the endoscope 301 is inserted into 

position sensor 226, is supplied to the control circuit 227 the stomach 303 through the mouth and the food passage, 

incorporated in the controller 227. The control circuit 227 45 and the insertion section 308 of the surgical manqmlator 302 

changes the orientation of the 3D scope 225 in accordance is inserted into the stomach 309 through a trocar 306 set in 

with the displacement which the position sensor 227 has an opening incised in, for example, the abdominal wall 305. 

detected. A bending tube 309 is connected to the distal end of the 

The master manipulators 228a and 2286 are connected to insertion section 304 of the endoscope 301, and a distal<nd 

the displacement calculating circuit 21 6A of the controller 50 unit 310 is attached to the bending tube 309. An operation 

207. The control circuit 227 controls the surgical manipu- section 311 is connected to the proximal end of the endo- 

lator 224a in accordance with the displacement of the scope 301. The operation section 311 contains an actuator 

manipulator 224a, which the circuit 216A has calculated (not shown) for driving the ending mbe 309 and the distal- 

from the position designated by the master manipulator end unit 310 under the control of a controller (not shown). 

228a, and the position of the object, which the circuit 216 55 As shown in FIG. 37A, the distal-end unit 310 has an 

has calculated. Similarly, the control circuit 227 controls the observation window 312, an illumination window 313 and 

surgical manipulator 2246 in accordance with the displace- an instrument channel 314. A hollow cylindrical hood 315 is 

mcnt of the manipulator 2246, which the circuit 216A has mounted on the circumferential surface of the distal-end unit 

calculated from the position designated by the master 310. A permanent magnet 316 is fastened to the inner 

manipulator 2286, and the position of the object, which the 60 ciromifercntial surface of the hood 315. A laser probe 317 

circuit 216 has calculated. extends through the instrumeiit channel 314, with its distal 

While wearing the HMD 208A and looking at the virtual end portion protruding outward from the distal-end unit 310. 

image of the object displayed by the HMD 208A, a surgeon Instead of the laser probe 317, an instrument such as a 

210 operates the master manipulators 228a and 2286, cautery knife, a forceps or a medidne-applying tube may be 

whereby the controller 207 remote-controls the surgical 65 inserted through the instrument channel 314. The insertion 

manipulators 224a and 2246. To be more specific, the section 304 of the endoscope 301 may be inserted into the 

control circuit 227 receives the information which the sense stomach 303 by using a manipulator. 
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The surgical maDipuktor 302 comprises a man^)ulator 
body 308 and a hollow cylindrical cap 320. The manipulator 
body 308 has joints 308a and connected at its proximal end 
to a base 318 which is secured to the floor. The cap 320 is 
fastened to the distal end of the manipulator body 308. The 
cap 320 has an inner diameter slightly larger than the outer 
diameter of the hood 315 of the endoscope 301, and can 
therefore be slidably mounted on the outer circumferential 
surface of the hood 315. 

Attached to the cap 320 is a Hall clement 321 for 
detecting the magnetic field emanating from the permanent 
magnet 316. The Hall element 321 can therefore detect the 
position of the distal-end unit 310 of the endoscope 301, the 
unit 310 and the distal ^d portion 319 of the manipulator 
302, both inserted in the abdominal cavity 307, and which 
are partitioned by the stomach wall 322 and cannot be seen 
from outside. The permanent magnet 316 and the Hall 
element 321 constitute a unit for detecting the position of the 
distal-eiul unit 310. 

The surgical manipulator 302 further comprises an actua- 
tor (not shown) for driving the manipulator body 318. The 
actuator is controlled by a controller (not shown). 

As described above, the cap 320 is slidably mounted on 
the outer ctrcumfetential siui^ of the hood 315. The hood 
315 and the 320 can therefore be moved toward each 
other, to damp the stomadi wall 322. In other words, the 
hood 315 and the cap 320 constitute a unit for clamping the 
stomach wall 322. This unit can damp a diseased part 323 
(e.g., a malignant tumor) of the stomach wall 322 or a 
portion of the wall 322 ambient to the diseased part 323. 
While clamped between the hood 315 and the cap 320 and 
held firmly, the diseased part 323 can be irradiated with a 
laser beam emitted &om the laser probe 317 inserted in the 
instrument channel 314 of the endoscope 301. Alternatively, 
while the diseased part 323 is held firmly, a cautery knife 
may be manipulated to incise the diseased part 323, a 
forceps may be used to extract a tissue from the part 323, or 
a medicine-applying tube may be directed to the part 232 to 
apply a medicine thereto. 

The fourteenth embodiment of this invention, which is a 
surgical manipulator system, will be described with refer- 
ence to FIGS. 39, 40A, 40B and 41. 

FIG. 39 is a perspective view showing the aorta 330, 
explaining how the surgical manipulator system is operated 
to implant an artifidal blood vessel 333 in the dissecting part 
332 of the aorta 330, thereby to remedy dissecting aneu- 
rysm. 

As shown in FIG. 39, the surgical manipulator system 
comprises, among other things, an endoscope 334, two other 
endoscopes 343, and two two-arm micro-manipulators 344. 

Ibe endoscope 334 has an insertion section 336, a bend- 
ing section 337, a distal-end unit 338 and a balloon 342. The 
insertion section 336 is inserted into the aorta 330, passing 
through an incised part 335 of one of the two femoral 
arteries 331 which are continuous to the aorta 330. The 
bending section 337 is connected to the distal end of the 
insertion section 336. The distal-end unit 338 is attached to 
the distal end of the bending section 337. The unit 338 has 
an observation window 339, an illumination window 340, an 
instrument channel 341, and a permanent magnet (not 
shown). The balloon 342 is mounted on the distal-end unit 
338 as iUustrated in FIGS. 40A and 40B. 

The insertion section 336 of the endoscope 334 is inserted 
in the artificial blood vessel 333. The balloon 342 is inflated, 
pressing the distal end portion of the blood vessel 333 from 
within. The artificial blood vessel 333 is thereby held 
steadfastly. The artifidal blood vessel 333 consists of two 
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rigid end portions 333a and a soft intermediate portion 333f>. 
The end portions 333a are made of bioceramics such as 
hydroxy apatite, p-TCP or the like. Ibe intermediate portion 
3336 is made of Goatex, Duclon or the like. Either end 
portion 333a has an annular groove 333c in its outer 
circumferential surface, as is best shown in FIG. 41. 

The endoscopes 343 are inserted via trocars (not shown) 
into the body cavity in which the aorta 330 and the femoral 
arteries 331 are located. Similarly, the two-arm micro- 
manipulators 344 are inserted into the body cavity through 
trocars (not shown). The endoscopes 343 are essentially 
identical in structure to the endoscope inserted in the aorta 
330. Hiey have a Hall element (not shown) each in the 
distal-end unit 338. Each two-arm micro-manipulator 344 
comprises two arms and two holders 345 connected to the 
distal ends of the arms, respectively. Hie holders 345 have 
a visual sensor (not shown) each. 

It will be explained how a surgeon operates the fourteenth 
embodiment in order to implant the artificial blood vessel 
333 in the aorU 330. 

First, the surgeon mounts the artificial blood vessel 333 on 
the insertion section 336 of the endoscope 334. The balloon 
342 is inflated, pushing the distal end portion 333a of the 
blood vessel 333 outwards from within. The artificial blood 
vessel 333 is thereby held steadfastly. 

Next, the surgeon inserts the insertion section 336 into the 
lefi femoral artery 331 through the indsed part 335 of the 
left femoral artery 331. He or she fiirther inserts the section 
336 through the dissecting part 332 of the aorta 330 imtil the 
distal-end unit 338 readies a position a little above the 
dissecting part 332 of the aorta 330. As a result of ttus, the 
artifidal blood vessel 333 seals the dissecting part 332. 

Further, the surgeon inserts the endoscopes 343 and the 
two-arm micro-man^ulators 344 into the body cavity 
through the trocars (not shown). The surgeon moves the 
endoscopes 343 toward the end portions 333a of the artifi- 
cial blood vessel 333, respectively. While observing the 
aorta 330 through the endoscopes 343, the surgeon operates 
the two-arm micro-manipulators 344, wrapping sutures 346 
around those portions of the aorta 330 \^ch are mounted on 
the end portions 333a of the artifidal blood vessel 333 and 
tying the sutures 346. The artificial blood vessel 333 is 
thereby fixed in the aorta 330. 

More precisely, either two-arm micro-manipulator 344 
winds a suture 346 around the aorta 330 and ties it, fastening 
the artifidal blood vessel 333 to the dissecting part 332 of 
the aorta 330. The two-arm micro-manipulator 344 auto- 
matically performs this sequence of complicated operations 
in accordance with a computer program. Either portion of 
the aorta 330 which has been tied with the suture 346 is fit 
into the annular 333c fomied in the end portion 333a of the 
artifidal blood vessel 333. The blood vessel 333 is prevented 
firom slipping down in the aorta 330. 

Once the artificial blood vessel 333 has been implanted in 
the aorta 330, its outer circumferential surface is covered 
with the inner surface of the aorta 330. Its end portions 333a, 
if made of P-TCP, will be gradually absorbed into the inner 
surface of the aorta 330, and will cease to exist. There will 
leave in the aorta 330 only the soft intermediate part 333fc 
which is as flexible as the aorta 330. 

The implantation of the artificial blood vessel 333 
described above, which is performed by using endoscopes, 
is totally transcutaneous. This surgery is little invasive and 
ensures fast recovery of the patient. 

In the thirteenth and fourteeoth embodiments, as indicated 
above, two medical instmments are located in and outside a 
body tube, respectively, in order to observe the interior and 
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exterior of the body tube and to achieve treatment on the 
body tube. The second medical instrument is moved by 
means of an actuator toward the target part of the body tube. 
Then, both medical instruments are manipulated, treating the 
target part of the body tube within a short time. 

The fifteenth embodiment of this invention will be 
described with reference to FIGS. 42, 43, 44A to 44C and 
45. 

FIG. 42 schematically illustrates the surgical manipulator 
system according to the fifteenth embodiment. Shown in 
FIG. 42 are an operating table 401 and the patient 402 lying 
on the back on the table 401. Parallel bedside rails 403 are 
laid on the sides of the operating table 401. The surgical 
manipulator systm comprises a first slave manipulator 
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displayed by means of a TV monitor. Further, the CCD 
camera 413 may be replaced by a unit comprised of a camera 
secured to the proximal end of the insertion section 406a and 
an optical image-transmitting device a relay lens, or may be 
replaced by a stereoscopic tmit comprised of two cameras 
secured to the proximal end of the insertion section 406a and 
two optical image-transmitting devices such as relay lenses. 

The medical instrument 404 has an insertion section 404a. 
AUached to the distal end of the insertion section 404a is a 
forceps 414 for holding a tissue or an organ. Instead of the 
forceps 414, a tool such as a knife^ a suture applicator or the 
like, may be attached the distal end of the insertion section 
404a. Cocmected to the proximal end of the insertion section 
404a is an actuator (not shown) for opening and closing the 



(hereinafter called "surgical manq>ulator 405"), a second 15 forceps 414. 

slave manq)ulator (hereinafter called "observation manipu- The couplers 405c and 407c are identical in structure, 

lator 407). The surgical manipulator 405 is slidably mounted Only the coupler 407c of the observation man^julator 407 

on the left bedside rail 403, and the observation manq)ulator will be described with reference to FIG. 43. 

407 is slidably mounted on the right bedside rail 403. As shown in FIG. 43, the distal end portion 415 of the 

The surgical manipulator 405 has a first shaft 405a which 20 second shaft 4076 of the man^)uUtor 407 has an axial hole 

can rotate and move vertically, a second shaft 4056 which is 416. In this hole 416, a shaft 418 on a bearing 417 is 

connected to the upper end of the first shaft 405a and can rotatably supported. Formed integral with the shaft 418 is a 

move horizontally, and a coupler 405c secured to the distal forked bearing holder 419. The holder 419 has a bearing 420 

end of the second shaft 456. Similarly, the observation having an axis which crosses the axis of the bearing 417 at 

manipulator 407 has a first shaft 407a whidi can route and 25 right angles. The bearing holder 419 supports a joint shaft 

move vertically, a second shaft 4076 which is connected to 421, allowing the shaft 421 to rotate freely. Thus, the joint 

the upper end of the first shaft 407a and can move shaft 421 can rotate with two degrees of freedom at point d 

horizontally, and a coupler 407c secured to the distal end of ^txe it intersects with the axis of the insertion section 406a 

the second shaft 4076. The connector 405c holds a medical of the 3D endoscope 406. The joint shaft 421 has an axial 

instrument 404, whereas the connector 407c hokls a 3D 30 hole 422, in which the insertion section 406a is fitted and 

endoscope 406. secured by a nut (not shown). 

As shown in FIG. 42, the surgical manq>ulator system The operation of the surgical manq>ulator system accord- 
further comprises a first controller 408, a second controller ing to the fourteenth embodiment will now be explained, 
410, a master manipulator (operating means) 409, a HMD with reference to FIGS. 44A to 44C and the flow chart of 
412, and a 3D digitizer 411. The surgical manipulator 405 is 35 FIG. 45. 

connected to the first controller 408, which in turn is As can be understood from FIGS. 44A to 44C, the 

cormected to the master manipulator 409. The observation position of the lower end 436 of the coupler 405c of the 

manipulator 407 is cormected to the second controller 410, surgical manQ)ulator 405 is known from the positions of 

which in turn is connected to the digitizer 411. The 3D TCPs on the axes a, b and c and the sizes of the second shaft 

digitizer 411, which is a position sensor, is mounted on the 40 4056. The same is true of the observation manipulator 407. 

HMD 412. Alternatively, it may be direcUy put on a sur- First, the coupler 405c is moved, setting its lower end 436 

geon's head. Furthermore, the d^tizer 411 may be replaced into alignment with an opening 402a incised in the body 

by a joystick or a key switch. wall, in Step SI* (FIG. 45), as shown in FIG. 44A Then, in 

The first shaft 405a of the surgical manipulator 405 can be Step S2' (FIG. 45), the data representing the position of the 

driven vertically and routed by actuators (not shown) such 45 lower end 436, thus aligned with the opening 402a, is stored 



as electromagnetic motors. The second shaft 4056 of the 
manipulator 405 can be driven horizontally and rotated by 
actuators (not shown, either) such as electromagnetic 
motors. Hence, the shafts 405a and 4056 and the electro- 
magnetic motors constitute a drive mechanism for driving a 
so-called cylindrical coordinate type manipulator which has 
a vertical axis a, a roUtion axis b and a horizontal axis c. 
Similarly, the second shaft 407a of the observation manipu- 
lator 407 can be driven vertically and routed by actuators 
(not shown) such as electromagnetic motors. The second 
shaft 4076 of the manipulator 407 can be driven horizontally 
and rotated by actuators (not shown, either) such as elec- 
tromagnetic motors. Hence, the shafts 407a and 4076 and 
the electromagnetic motors constitute a drive mechanism for 
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as daU showing the position of the op^iirig 402a, in a 
memory incorporated in the controller 408. 

Next, in Step S3*, the surgical manipulator 405 is 
operated, moving the coupler 405c upwards, away from the 
opening 4Q2a, as ilhistrated in FIG. 44B. In Step S4', the 
medical instrument 404 is atUched to the coupler 405c, and 
the insertion section 404a of the instrument 404 is inserted 
into a body cavity, first through the axial hole 422 of the joint 
shaft 421 and then through the opening 402a — as illustrated 
in FIG. 44C. Then, the insertion section 404a is held in the 
axial hole 422 by means of a nut (not shown). 

Although not shown in FIGS. 44A to 44C, the 3D 
endoscope 406 is atUched to the coupler 407c in the same 
way as the medical instrument 404, and the insertion section 



driving a so-called cylindrical coordinate type manipulator 60 406a of the 3D endoscope 406 is inserted into the body 



which has a vertical axis a, a rotation axis b and a horizontal 
axis £. These drive mechanisms may be replaced by multi- 
joint manipulators. 

The 3D endoscope 406 has an insertion section 406a. A 
CCD camera 413 is attached to the distal end of the insertion 
section 406a. An image provided by the CCD camera 413 is 
displayed by the HMD 412. Instead, the image may be 
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cavity in the same way as the insertion section 404a of the 
instrument 404. 

In Step S5' (FIG. 45), the position of a TCP 437fl, such as 
the focal point of an objective lens located at the distal end 
of the insertion section 406a of the endoscope 406, is 
determined from the positions of the shafts 407a and 4076, 
the position and size of the opening 402a and the distance 
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between the TCP 437a and the opening 402a. In Step SS\ 
too, the position of an TCP 437B, such as the forceps 414 
attached to the distal end of the insertion section 404a of the 
medical instrument 404, can be determined £rom the posi- 
tions of the shafts 405a and 40Sbf the position and size of the 
opening 402a and the distance between the TCP 437b and 
the opening 402a. 

The positioD of the TCP 409a of the master man^ulator 
409, for exanq)l&, the center 411a of the digitizer 411 or the 
distal cod 409b of the master manipulator 409 is calculated 
by the controller 40S. In Step S6* (FIG. 45), the surgeon 
operates the master manipulator 409 such that the TCP 437a 
of the iiKertion section 404a and the TCP 437b of the 
insertion section 407b move to the TCP 409a of the master 
manipulator 409. 

Since the insertion section 404a is connected to the shaft 
405b by the coupler 405c having two degrees of freedom, 
and the insertion section 406a is coimected to the shaft 407b 
by the coupler 407c having two degrees of freedom, neither 
the medical instrument 404 nor the 3D endoscope 406 exerts 
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through the hole 429 and held firmly at the hole 429 by 
means of a nut (not shown). 

Since the distance L between the axes 433 and 435 is long, 
the distal end of the insertion section of the medical instru- 
ment or the 3D endoscope and the distal end of the shaft of 
the surgical or observation manipulator hardly interfere with 
each other. As a result, the TCP of the shaft 428 can move 
in a large space, and ultimately the TCP of the insertion 
section of the medical instrument or the 3D endoscope can 
move in a large space. As the surgeon turns his or her head, 
while looking at the 3DS image of the interior of a body 
cavity, di^layed by the HMD he or she wears, the distal end 
of the 3D endoscope is moved. While performing a trans- 
cutaneous endoscope surgery, the surgeon has such sense of 
immediacy that he feels as if he or she were in the body 
cavity. 

The eighteenth embodiment of the invention will be 
described with reference to FIG. 48 and FIGS. 49A and 49B. 

The eighteenth embodiment is a surgical manipulator 
system e^les a surgeon to observe an object in a body 



any excessive force at the opening 402a even if the patient 20 cavity and treat the object, without exerting an excessive 
moves. force at the openings incised in the body wall, through which 
In the conventional surgical manipulator system, each an observation manipulator and a surgical manipulator are 
manqjulator needs to have at least five degrees of freedom inserted into the body cavity. Any component similar or 
to move its TCP to a desired point in a body cavity, In the identical to that the fifteenth CTibodiment is designated by 
fifteenth embodiment, each man^ulator needs only three 25 the same reference numeral and will not described in detail, 
degrees of freedom to accomplish the same thing. The As shown in FIG. 49A, a bending section 440 is con- 
fifteenth embodiment can be simple in structure. nected to the distal end of the insertion section 404a of a 
Furthermore, since both manipulators used are hollow cylin- medical instrument 404. As shown in FIG. 49B, a bending 
diical ones having a cylindrical coordinate system, the secticm 440 is connected to the distal end of the insertion 
second shafts 405b and 407b of the manq)ulators protrude 30 section 406a of a 3D endoscope 406. Either bending section 
horizontally from the sides of the operating table 1, but for 440 has a bending mechanism (not shown). The bending 
only a short distance. In addition, since neither first shaft medianism has two degrees of freedom and can be driven by 
405aof the surgical manipulator 405 nor the first shaft 407a an actuator 441 contained in the proximal end of the 
of the observation manipulator 407 inclines sideways, there insertion section (404a, 406a). Wires arc used to transmit a 
is no obstacle to the surgeon or the nurses assiRting the 35 force from the actuator 441 to the bending mechanism. The 



surgeon. 

Ibe sixteenth embodiment of the present invention will be 
described with reference to FIG. 46. 

FIG. 46 shows a modification of the couplers 405c and 
407c. The modified coupler comprises a bearing 423 and a 40 
movable member 425. The bearing 423 has a hole whose 
inner surface is concave. The member 425 is fitted in the 
hole of the bearing 423 and has three degrees of freedom. 



insertion sections 404 and 406a have a shaft 442 each. The 
shaft 442 is rotated around its axis by a force transmitted 
from the actuator 441 by a traasmitting means 443 such as 
gears, a timing belt or the like. 

In the conventional surgical manipulator system, each 
manipulator needs to have at least eight degrees of freedom 
to move the TCP of an insertion section to a desired point in 
a body cavity. In the eighteenth embodiment, eadi manipu- 
lator needs only six degrees of freedom to accomplish the 



cq)able of routing around point e. The movable member 

425 has a hole 424. As in the fifteenth embodiment, the 45 same thing. The eighteenth embodiment can be simple in 

insertion section 404a of a medical instrument 404 or the structure. 

insertion section 406a of a 3D endoscope 406 is guided Furthermore, since both manipulators used are hollow 

through the hole 424 and held firmly at the hole 424 by cylindrical ones, the second shafts 405b and 407b of the 

means of a nut (not shown), manipulators protrude horizontally from the sides of an 

The seventeenth embodiment of this invention will be 50 operating table 401, but for only a short distance. In 



described with reference to FIG. 47. 

FIG. 47 shows a modification of the couplers 405c and 
407c. As shown in FIG. 47, the distal end portion 415 of a 
shaft has a hole 426 extending at right angles to the axis of 
the shaft. A bearing 427 is provided on the inner surface of S5 
the hole 426. A shaft 428 is rotatably supported by the 
bearing 427. A holder 430 has two holes 431 and 429 which 
extend at right angles to each other. A bearing 432 is 
moimted on the inner surface of the hole 431. A shaft made 



addition, since neither first shaft 405a of the surgical 
manipulator 405 nor the first shaft 407a of the observation 
manipulator 407 inclines sideways, there is no obstacle to 
the surgeon or the nurses assisting the surgeon. 

As described above, in the fifteenth to eighteenth 
embodiments, as the surgeon operates the master 
manipulator, the slave manipulators are operated in the same 
maimer, moving the TCPs of the surgical devices to desired 
points in a body cavity. Moreover, as the surgeon turns his 



integral with the shaft 428 and extending at right angles 60 or her head, while looking at the 3D image of the interior of 



thereto is inserted in the hole 431 of the holder 430 and 
supported by the bearing 432. The axis 433 of the shaft 428 
intersects with the axis 434 of the holder 430 at right angles, 
but docs not intersect with the axis 435 of the insertion 
section of a medical iustriiment or a 3D endoscope. 

As in the fifteenth embodiment, the insertion section of 
the medical instrument or the 3D endoscope is guided 
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a body cavity, displayed by the HMD he or she wears, the 
distal end of the 3D endoscope is moved. While peirforming 
a transcutaneouis endoscope siirgery, the surgeon has such 
sense of immediacy that he or she feels as if be or she were 
in the body cavity. 

The nineteenth embodiment of the present invention will 
be described with reference to FIGS. 50 to 57, HGS. 58A 
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and 58B, and FIG. 59. This embodiment is a suigical freedom. The holder section 524 contains a rotary mecha- 

manipulator system which comprises two manipulators 503 nism for rotating the medical instrument 520 in the direction 

and 504, a surgical section 531 and an observation section of arrow A4. 

533. The medical instrument 520 comprises a grip 520a, an 

As shown in FIG. 50, the surgical section 531 comprises 5 insertion section 520fe and a holding forceps 517. The grip 

a master manipulator 501, a motor control unit 502 and a 520^ is connected to the proximal end of the ins^on 

surgical manipulator 503. The master man^ulator 501 is section 5206 and contains a drive mechanism. The insertion 

operated by a surgeon 516. The motor control unit 502 section 5206 can be inserted into a body cavity and bent by 

rmote-controls the surgical man^ulator 503 in accordance operating the drive mec^iaoism contained in the grip 520a. 

with information output from the master manipulator 501 lO The holding forceps 517 is connected to the distal end of the 

and representing the position and orientation of the sur- insertion section 5206 and can be opened and closed by 

gcon's hand. A medical instrument 520 is attached to the operating the drive mechanism. 

distal end of the surgical manipulator 503. The iostrument As ^own in HG. 53, the observation manipulator 504 

520 can be bent and has an insertion section 5206. The comprises an aim section 519, the 3D scope 523 and a holder 

insertion section 5206 is inserted into an abdominal cavity 15 section 524. The arm section 519 and the holder section 524 

514 of the patient laying on the back on an operating table are identical to those of the suigical manipulator 503. The 

512 — through a trocar 530 set in an opening incised in the 3D scope 523 is removably attached to the distal end of the 

abdominal wall of the patient. As shown in FIG. 51, the holder section 524. 

operating table 512 has two parallel bedside rails 513. The The 3D scope 523 comprises a gpp 523a, an insotion 

master manq»ulator 501 and the surgical manq>ulator 503 are 20 section 5236 and an imaging unit 5Z2. The grip 523a is 

removably connected to two supports 515 secured to the connected to the proximal end of the insertion section 5236 

bedside rails 513. contains a drive mechanisHL The insertion section 5236 

The observation section 533 comprises an observation can be inserted into a body cavity and bent by operating the 

manipulator 504, a manipulator driver 505 and a motor drive mechanism contained in the grip 523a. The imaging 

control unit 506. The motor control unit 506 remote-controls 25 unit 522 is attached to the distal end of the insertion secdon 

the observation manipulator 504 in accordance with control 5236. 

signals supplied firam the manipulator driver 505. A 3D FIG. 54 ilhistrates a mechanism for connecting a medical 

scope 523 wfaidi can be bent is connected to the distal end instrument 520 or the 3D scope 523 to the holder section 

of the observation manipulator 504. The 3D scope 523 has 524. As shown in FIG. 54, the holder section 524 has a tet 

an insertion section 5236, The insertion section 5236 is 30 member 524a which is diaped like a flaoge and which 

inserted into an abdominal cavity 514 of the patient via a extends in the lengthwise direction of the holder section 524. 

trocar 530 set in an opening incised in the abdominal wan of The first member 524a has a hole 529, through whidi the 

the patient As shown in FIG. 53, the observation manqm- insertion section 5206 of the medical instrument 520 or the 

lator 504 is removably connected to a supports 515 secured insertionsection5236of the 3D scope 523 can pass. Ascrew 

to the bedside rail 513. The manipulator driver 505 may be 35 535 is formed on the proximal end portion of the insertion 

a joystidc, a 3D mouse, a 3D digitizer, a manipulator or the section 5206, and a screw 535 is formed on the proximal end 

lite. portion of the insertion section 5236. Hence, when the 

The 3D scope 523 has a pair of imaging elements (not insertion portion 5206 or 5236 is guide through the hole 529 

shown) which can generate image signals representing the of the first member 524a and a connecting nut 521 is set in 

imageof the interior of the abdominal cavity 514. The image 40 saew engagement with the proximal end portion of the 

signals generated by the imaging elements are supplied via insertion section 5206 or 5236, the instrument 520 or the 3D 

two CCUs 507a and 5076 to image processing devices 508a scope 524 is fastened to the holder 524. When the connect- 

and 5086, respectively. The signal output from the device ing nut 521 is released from the screw engagement vrith the 

508a is siq)plied to a TV monitor 511, which displays an proximal end portion of the insertion section 5206 or 5236, 

image of the interior of the body cavity. The signals output 45 the instrument 520 or the 3D scope 523 is easily discon- 

from the devices 508a and 5086 are supplied to a TV nected from the holder 524. 

monitor 511 and two displays 510a incorporated in a HMD ETG. 55 shows a mechanism for connecting the holder 

510. The displays 510a cooperate to provide a 3D image, section 524 and the arm section 519. As already indicated, 

which the sxu^con 516 who wears tbe HMD 510 can sec. A the holder section 524 is removably attached to the arm 

light source 509 is connected to the observation manipulator so section 519 by the universal joint 518 ^ich is connected to 

504, whereby light can be supplied into the abdominal cavity the distal end of the second drive shaft 519c. The holder 

514 through the manipulator 504. section 524 can easily be coupled to the universal joint 518 
As shown in FIG. 52, the surgical manipulator 503 by means of a single connecting nut 525. 

comprises an arm section (main body) 519, a holder section As shown in FIG. 55, the universal joint 518 has a 
524, and the medical instrument 520. The arm section 519 55 projection 518a having a flange at its distal end. The holder 

comprises a fixed shaft 519a. a first drive shaft 5196 and a section 524 has a second member 5246 which has a screw 

second drive shaft 519c. The fixed shaft 519a has a fastener 528 cut in its circumferential surface. When the connecting 

515 secured to the bedside rail 513. The first drive shaft 5196 nut 525 is set into engagcmwit with the screw 528 of the 
can move vertically (in the direction of arrow Al) and rotate second member 5246, with the conneaing nut 525 mounted 
around its axis (in the direction of arrow A2). The second 60 on the flange of the projection 518a, the holder section can 
drive shaft 519c can move horizontally (in the direction of be attached to the arm section 519. 

arrow A3). The medical instrument 520 is removably When the connecting nut 525 is released from engage- 
attached to the holder section 524. ment with the screw 528, the holder section can easily be 

The holder section 524 is removably attached to the disconnected from the arm section 519. 
second drive shaft 519c of the arm section 519 by means of 65 The projection 518a of the universal joint 518 is made of 

a universal joint 518 which is connected to the distal end of electrically insulating material, and the holder section 524 is 

the second drive shaft 519c and which has some degrees of sealed in watertight fashion. 
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FIG. 56 shows the drive mechanism contained in the 
medical instrument 520 and the rotary medianism contained 
in the bolder section 524. FIG. 56 shows the case where the 
instrument 520 is attached to the holder section 524 by 
means of the connecting nut 521. 

The rotary mechanism contained in the holder section 524 
comprises an electric motor 545, a reduction unit 546, an 
encoder 550, a first pulley 547, a second puUcy 548, and an 
endless belt 549. The motor 545, the reduction unit 546, the 
encoder 550 and the first pulley 547 arc aligned substantially 
in a straight line. The reduction unit 546 is coupled to one 
end of the shaft of the motor 545, and the encoder 550 is 
connected to the other end of the motor 545. Hie first pulley 
547 is mounted on the output shaft of the reduction unit 546. 
The components 545, 546, 550 and 547 are aligned in a 
substantially straight line. The second pulley 548 is a hollow 
cylinder secured to the first member 524a of the older 
section 524. The endless belt 549 is wrapped around the first 
and second pulleys 547 and 548. The rotation of the motor 
545 can be transmitted to the medical instrument 520 fitted 
in the second pulley 548, by the reduction unit 546, the first 
pulley 547, the belt 549 and the second pulley 548. When the 
motor 545 is driven, the second pulley 578 is rotated, 
rotating the medical instrument 520. 

As indicated above, the insertion section 5202> of the 
medical instrument 520 has the holding forceps 517 fixed to 
its distal end. The insertion section 5206 coiiq}rises a long 
pipe 538, a bending mechanism 537 and the forcq)s 517. 
The bending mechanisan 537 has a plurality of segments 
which are coupled together, end to end, each rotatable with 
respect to either atflacent one. A pair of wires 539 and 540 
are fastened, at one end, to the bending mechanism 537. The 
wires 539 and 540 are fastened, at the other end, to a first 
linear actuator 541 which is contained in the grip 52(h and 
set in engagement with a pulley 561 incorporated in the grip 
520a. When the linear actuator 541 is driven abng its axis, 
one of the wires 539 and 540 is pulled, whereas the other 
wire is sladcened, or vice versa. The segments of the 
bending mechanism 537 are thereby rotated, whereby the 
bending mechanism 537 turns upwards or downwards. Con- 
nected to the distal end of the linear actuator 541 is a 
free-lock mechanism 553, which will be described later. 

A rod 542 is connected, at one end, to a link mechanism 
for opening and closing the holding forceps 517. The other 
end of the rod 542 is connected by a flexible rod 543 to one 
end of an operation rod 544. The other end of the operation 
rod 544 is connected to a second linear actuator 551 which 
is contained in the grip 520a. Therefore^ when the second 
linear actuator 551 is driven along its axis, the rod 542, the 
operation rod 544 and the flexible rod 543 are thrust bac^ 
and forth, actuating the link mechanism. The holding for- 
ceps 517 is thereby opened and closed. 

Tlie flexible rod 543 comprises an inner cable, a spring, a 
blade, and an outer casing. The inner cable is a strand of 
stainless wires connected to the operation rod 544, for 
transmitting a push and a pull to the operation rod 544. The 
spring is made of stainless steel, holding and protecting the 
inner cable. The blade covers the spring. The outer casing, 
which is made of ETFE, is sheath covering the blade. 

Tbe first linear actuator 541, which pulls and slacken the 
wires 539 and 540, comprises an electric motor 554, an 
encoder 555, a coupling 556, a feed screw 557 and a sliding 
mechanism 550. The encoder 555 is mounted on the shaft of 
the electric motor 554. The feed screw 557 is coupled to the 
motor 554 by the coupling 556. The sliding mechanism 550 
is set in screw engagement with the feed screw 557 and 
connected to the wires 539 and 540. 
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As illustrated in FIG. 57, the sliding mechanism 550 
comprises a nut 558, a nut housing 559 and a sliding shaft 
564. The nut is mounted on the screw 557 and put in screw 
engagement therewith. The nut housing 559 supports the nut 

s 558. The sliding shaft 564 enables the nut housing 559 to 
slide along the feed screw 557. A linear bush 563 extends 
parallel to the nut bousing 559, in order to prevent the nut 
housing 559 from rotating. Hie sliding mechanism 550 is 
located, in its entirety, in a housing 560 whidi supports and 

10 holds the feed screw 557. A pulley 561 is rotatably con- 
nected to the housing 560. The wire 539 fastened to the 
bending medianism 537 is fixed at its end to the nut housing 
559 by a wire fastener 562. The wire 540, which is fastened 
to the bending mechanism 537, too, is wrapped around the 

15 pulley 561 and fixed at its end to the nut housing 559 by a 
wire fastener 562. 

The second liner actuator 551 is identical to the first linear 
actuator 541, except for two respects. First, it has neither a 
wire fastener nor a pulley to fix the wire 539 or 540. Second, 

20 the operation rod 544 is secured to the bending mechanism 
537. 

The firee-lock mechanism 553, which is connected to the 
distal end of the first linear actuator 541, will be described 
in detail, with refererx^ to FIGS. 58A and 58B. As shown in 

25 these figures, the mechanism 553 comprises a compression 
spring 566, a spring holder 567, a sliding rod 569, a contact 
plate 565, and a guide member 568. The oompression spring 
566 is wound aroumd the distal end portion of the housing 
560. The spring holder 567 holds the compression spring 

30 566, pushing the spring 566. The sliding rod 569 is con- 
nected to the spring holder 567 and can protrude outside the 
housing 560 through a hole 560a made in the distal end of 
the housing 560. The contact plate 565 is secured to the 
distal end of the sliding rod 569 and can abut on the first 

35 member 524a of the holder section 524 when biased by the 
compression spring 566. The guide member 568 guid^ the 
housing 560 in a predetermined direction. 

Connected together by the sliding rod 569, the contact 
plate 565 and the spring holder 567 can move together, back 

40 and forth, along the axis of the medical instrument 520. 
Guided by the guide member 568, the housing 560 can move 
back and forth, also along the axis of the medical instrument 
520. 

The 3D scope 523 and the holder section 524 will be 

45 described in detail, with reference to FIG. 59. 

As can be seen from FIG. 59, the 3D scope 523 contains 
a bending mechanism, and the hold^ section 524 a rotary 
mechanism. In the condition shown in FIG. 59, the 3D scope 
523 is fastened to the holder section 524 by the nut 531, as 

50 described above. The holder section 524 and the rotary 
mechanism contained in the section 524 are identical to 
those which have been described in detail with reference to 
FIG. 56. The components of the holder section 524 and the 
rotary mechanism are therefore designated at the same 

55 numerals as used in FIG. 56 and will not be described in 
detail. 

The bending mechanism contained in the grip 523a of the 
3D scope 523 is identical to the drive mechanism contained 
in the medical instrument 520, except that it has no second 
60 actuator to drive an operation rod. The bending mechanism 
will be described briefly. 

As indicated above, the insertion section 523f) of the 3D . 
scope 523 has the imaging imit 522 attached to its distal end. 
The insertion section 523b comprises a long pipe 538, a 
65 bending mechanism 537 and the imaging unit 522. 

The imaging unit 522 has two CCD cameras 581, each 
comprising a CCD sensor and an optical system. A cable 579 
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passes through the pipe 538 and electrically connects the 
CCD cameras 581 to a CCU 507 of the same type as those 
shown in FIG. 50. 

The bending mechanism 537 has a plurality of segments 
which are coupled together, end to end, each rotatable with 
respect to either adjacent one. A pair of wires 539 and 540 
are fastened, at one end, to the bending mechanism 537. The 
wires 539 and 540 are wrapped around a pulley 561 pro- 
vided in the grip 523^2 and fastened, at the other end, to a 
linear actuator 541 contained in the grip 523a. The linear 
actuator 541 is identical in structure to the first linear 
actuator 541 incorporated in the medical instrument 520. 
When the linear actuator 541 is driven along its axis, one of 
the wires 539 and 540 is pulled, whereas the other wire is 



10 



and the bending mechanian of the instrument 520 in accor- 
dance with the information supplied from the master 
manipulator 501. The holding forceps 517 fixed to the distal 
end of the surgical instrument 520 is thereby controlled to 
assume the position and orientation represented by the 
information. 

The surgical manipulator 503 and the observation 
manipulator 504 may interfere with each other when the 
surgeon moves the master manipulator 501 too much, even- 
tuaHy positioning the stu^cal manipulator 503 too close to 
the 3D scope 523. Alternatively, the arm sections 519 of the 
manipulators 503 and 504 may be so positioned that neither 
the forceps 517 nor the 3D scope 523 can move to desired 
points in the body cavity. Either problem can be solved. 



slackened, or vice versa. The segments of the bending 15 merely by removing the instrument 520 and the 3D scope 



mechanism 537 are thereby rotated, whereby the bending 
mechanism 537 turns upwards or downwards. Connected to 
the distal end of the linear actuator 541 is a free-lock 
mechanism 553 whidi is identical in structure to the mecha- 
nism 553 used in the medical instrument 520. 

As can be understood from the preceding paragraph, tbe 
3D scope 523 differs in structure from the medical instru- 
ment 520, only in that imaging unit 522 is used in place of 
the holding forceps 517. This may be best illustrated in FIG. 



523 from the first and second holder sections 524, 
respectively, and by switdiing the positions of the instru- 
ment 520 and the 3D scope 523. 
After the surgery has been performed, the holder section 
20 524 and the medical instrument 520 are removed firom the 
aim section 519 and sterilized along with the 3D scope 523. 
The arm section 519 can be sterilized as well, but should 
better be covered with a piece of sterilizing cbth. The 
sterilizing doth is taken off the arm section 519 and dis- 



57 which shows imaging unit 522 along with the holding 25 carded after the section 519 has been thoroughly ^erilized. 



forceps 517. 

The surgical manipulator system equipped with the 
manipulators 503 and 504 described above will now be 
explained. 

Before the initiation of the surgery, the medical instru- 
ment 520 and holder section 524 of the surgical manq)ulator 
503 have yet to be attached to the arm section 519. Both the 
instrument 520 and the holder section 524 are already 
sterilized. 



The operations of the surgical manq>ulator 503 and the 
obsenration manqtulator 504 will be plained separately, to 
facilitate understanding. 
It will first be described how to operate the surgical 
30 manipulator 503. Before used in surgery, the medical instru- 
ment 520 remains detached from the holder section 524 and 
sterilized, as described above. The instrument 520 is main- 
tained watertight by means of an O-ring so that it may be 
sterilized well The surgical man^nilator 503 isvpositioned 



To start the surgery, the arm section 519 is secured to the 35 with its TCP located near the object of surgery. Then, the 



operating table 512 by mears of the fastener 515. Next, the 
holder section 524 is attached to the projection 518a of the 
universal joint 518. More specifically, as shown in FIG. 55, 
the coimecting nut 525 is set into engagement with the screw 
528 of the holder section 524, while keeping the mit 525 in 40 
engagement with the flange of the 518a. Then, as shown in 
FIG. 54, the connecting nut 521 is set into engagement with 
the screw 535 of the insertion section 520^, with the section 



insertion section 5206 is inserted into the trocar 530, while 
held in the hole 539 of the holder section 524. Thereafter, the 
medical instrument 520 is attached to the holder section 524 
by means of the connecting nut 521. 

In the holder section 524, the rotation of the electric motor 
545 is transmitted to the second puHey 548 by the reduction 
unit 546 and the first pulley 547, the endless belt 549, as can 
be understood from FIG. 56. As the second pulley 548 is 
rotated, the medical instnmient 520, which is fastened to the 



520f) inserted in the hole 529 of the holder section 524. As 

a result, the medical instrument 520 is attached to the holder 45 connecting nut 521, is rotated as a whole, 

section 524. The holding forceps 517 fixed to the distal end of the 

Similarly, the 3D scope 523 and the holder section 524 of surgical instrument 520 is driven by the first linear actuator 

the observation manipulator 504 have yet to be attached to 541. The forceps 517 holds the object when closed, and 

the arm section 519 before the initiation of the surgery. Both releases the object when opened. The bending mechanism 

the 3D scope 523 and the holder section 524 are already 50 537 is bent upwards or downwards by the hnear actuator 



sterilized. To start the surgery, the arm section 519, the 
holder section 524, and the 3D scope 523 are sequentially 
secured to the operating table 512. 

Hie surgeon operates the master manipulator 501, while 
looking at the image of the interior of a body cavity, which 55 
is displayed by the TV monitor 511 or the HMD 510. As the 
surgeon operates the master manipulator 501, the surgical 
manipulator 503 is operated in the same way, performing the 
surgery on the object in the body cavity. To stale more 
specifically, the master manipulator 501 detects the position 
and orientation of the surgeon's hand, and supplies the 
information representing the position and orientation of the 
hand to the motor control unit 502. The unit 502 remote- 
controls the surgical manipulator 503 in accordance with the 
information. To be more precise, the unit 502 controls the 
first and second drive shafts 5196 and 519c of the arm 
section 519, the rotary mechanism of the holder section 524 



60 
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551. The free-lock mechanism 553 is operated by tightening 
the cormecting nut 521, thereby attaching the medical instru- 
ment 520 to the holder section 524. Namely, the insUiiment 
520 is set into locked condition in vAnch. the drive force of 
the linear actuator 541 is transmitted directly to the bending 
medianism 537. When the cormecting nut 521 is loosened, 
the instrument 520 is set into unlocked condition in which 
the bending mechanism 537 can be easily bent by external 
forces. 

The operation of the free-lock mechanism 553 will be 
explained below. 

The connecting nut 521 is set in engagement with the 
screw 535 of the insertion section 5206, with the section 
5206 guided through the hole 529 of the holder section 524, 
as shown in FIG. 58B. Then, the pulley 548 of the holder 
section 524 contacts the contact plate 565. As the nut 521 is 
tightened, the contact plate 565, the sliding rod 569 and the 
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Spring holder 567 can move together, to the right, or toward 
the proximal end. As the spring holder 567 is moved to the 
right, it compresses the compression spring 566. The spring 
566 urges the housing 560 of the first Hncar actuator 541 to 
slide to the right on the guide member 568. The compression 5 
spring 566 does not make the actuator 541 to slide so. 
Rather, its force increases the tension on the wires 539 and 
540. This is because the housing 560 is fastened to the wires 

539 and 540 by the pulley 561 and the nut housing 559. In 
other words, the spring 566 tightens the wires 539 and 540. lO 
Both wires 539 and 540 therefore transmit all drive force of 
the linear actuator 541 to the bending mechanism 537. The 
mechanism 537 is thereby bent reliably. Namely, the bend- 
ing mechanism 537 is set in locked state by the linear 
actuator 541 and is bent in a desired manner. 15 

When the connecting nut 521 is tightened, fastening the 
instrument 520 to the holder section 524, as iUustrated in 
FIG. 58B, the compression spring 566 has but a anall bias, 
and the tension on the wires 539 and 540 is low. The bending 
mechanism 537 can be bent easily by an external force. In 20 
this condition, the compression spring 566 absorbs the drive 
force of the first linear actuator 541. No large force is applied 
to the bending medianian 537. The bending mechanism 537 
is in an unlocked state. 

The force of the compression spring 566 can be dianged 25 
by tightening or loosening the connecting nut 521, to thereby 
vary the tension on the wires 539 and 540. To put it another 
way, it is possible to adjust the tension on the wires 539 and 

540 by turning the connecting nut 521 in one direction or the 
other. 30 

While remaining in the locked state, the bending mecha- 
nism 537 is drivoi by the first linear actaator 541 in the 
foUowing way. 

As already described, the bending mechanism 537 is bent 
by pulling the wire 539 and slackening the wire 540, or vice 35 
versa. It is the first liocar actuator 541 (FIG. 57) that pulls 
and sladcen each of these wires 539 and 540. V/bea the 
electric motor 554 is driv«i, the feed screw 557 rotates, 
moving the nut housing 559 along with the nut 558. As the 
nut 558 is moved to the right in FIG. 57, the wire 539 is 40 
pulled to the right. At this time, the wire 540 is moved to the 
left since it is wrapped around the pulley 561. As a result of 
this, the bending mechanism 537 is bent upwards as illus- 
trated in FIG. 57. Conversely, when the nut 558 is pulled to 
the left, pulling the wire 539 to the left and tbc wire wire 540 45 
to the right, the bending mechanism 537 is bent downwards. 

When the second linear actuator 551 is driven in the same 
way as the first linear actuator 541, the nut bousing is moved 
to the left or the right. Then, the operation rod 544 is driven 
forward to open the holding forceps 517, or backward to 50 
close the holding forceps 517. 

It will now be described how to operate the observation 
manipulator 504 by using the HMD 510 only, not using the 
TV monitor 511. 

At first, the surgeon puts on the HMD 510. The left-eye 55 
image and the right-eye image, obtained by the CCD cam- 
eras 581, are supplied to the display SlOa through the cable 
579. The display 510a displays a 3D image of the interior of 
the body cavity, which the surgeon sees. The surgeon turns 
his or her head to see the object he or she intend to treat. The 60 
3D digitizer 510ft (i.e., the manipulator driver 505 shown in 
FIG. 50) detects the position and orientation of the surgeon's 
head and generates a signal representing the position and 
orientation of the head. In accordance with this signal, the 
motor control unit 502 controls the drive shafts 519fc and 65 
519c of the arm section 519, the rotary mechanism of the 
holder section 524 and the bending mechanism of the 3D 
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scope 523, thereby setting the imaging tmit 522 attached to 
the distal end of the 3D scope 523. This enables the surgeon 
to observe the object as if he or she were in the body cavity. 

The holder section 524 and arm section 519 of the 
observation manipulator 504 are identical to those of the 
surgical manipulator 503. Therefore, the rotary mechanism 
in the holder section 524 and the bending mechanism 537 in 
the 3D scope 523, and the free-lock mechanism 553 operates 
in the same way as their cotmteiparts of the surgical manipu- 
lator 503. 

More specifically, in the holder section 524, the rotation 
of the electric motor 545 is transmitted to the second pulley 
548 by the reduction unit 546 and the first pulley 547 and the 
endless belt 549. The second pxilley 548 is thereby rotated, 
and the 3D scope 523, which is fastened to the connecting 
nut 521, is rotated as a whole. 

The connecting nut 521 is set in engagement with the 
screw 535 of the insertion section 5236, with the section 
523ft guided through the hole 529 of the bolder section 524, 
as shown in FIG. 58A. Then, the pulley 548 of the holder 
section 524 contacts the contact plate 565. As the nut 521 is 
tightened, the contact plate 565, the sliding rod 569 and the 
spring holder 567 can move together, to the right, namely 
toward the proximal end. Thereafter, the same sequence of 
operations takes place as in the su^ical manipulator 503. 
Finally, all drive force of the linear actuator 541 is trans- 
mitted to the bending mechanism 537, and th^mechanism 
537 is thereby bent reliably. Namely, the bending mecha- 
nism 537 is set in lodced state by the Unear actuator 541. It 
is bent in a desired manner, against any external force 
applied to it 

When the connecting nut 521, which connects the 3b 
scope 523 to the holder section 524, is loosened as illustraied 
in FIG. 58B, the compression spring 566 has but a ^all 
bias, and the tension on the wires 539 and 540 is low.' The 
bending mechanism 537 can be bent easily by an external 
force. 

As explained above, the holder section 524 of either 
manipulator (503 or 504) can be detached from the universal 
jomt 518 of the arm section 519. Once detached from the 
universal joint 518, the holder section 524 can be sterilized 
with ease. Similarly, since the medical instrument 520 and 
the 3D scope 523 can be removed ficom the respective holder 
sections 524, they can be easily sterilized, particularly those 
portions which have contacted living tissues located in the 
body cavity. Furthermore, once detached from the arm 
sections 519, the holder sections 523, the instrument 520 and 
the 3D scope 523 can be stored in sterilized condition and 
can therefore be maintained in better conditions than other- 
wise. The distal end of the 3D scope 523 may get dirty or 
clouded in the course of the surgery. If this happens, the 3D 
scope 523 can be removed from the holder section 519, 
washed and sterilized, and attached back to the holder 
section 519 — within a short period of time. This serves to 
enhance the efficiency of surgery. 

Since the projection 518a of the universal joint 518 is 
made of electi-ically insulating material, it electrically insu- 
lates the joint 518 firom the holder section 524. An electric 
current would not flow to the arm section 519 from, for 
example, a cautery knife. 

Various kinds of medical tools, such as a holding forceps, 
an ablation forceps, a needle -holding forceps, a clamp 
forceps and a cautery knife, can be attached to the distal end 
of the instrument instrument 520. The surgeon can fast 
replace the tool of any kind, attached to the instrument 520, 
with another tool of any other kind in the midst of the 
surgery. 
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As the surgery proceeds, tbe surgical manipulator 503 and 
the observation manipulator 504 may interfere with each 
other. The interference can be eliminated, merely by switch- 
ing the instrument 520 and the 3D scope 523 in terms of their 
positions. This is possible simply because the arm sections 

519 of the manipulators 503 and 504 are identical in 
structure. The surgeon can have great freedom of observing 
and treating the object of surgery. That is, the surgical 
manipulator system has high operability. 

In the nineteenth embodiment, the medical instrument 

520 and the 3D scope 523 are coimected to and removed 
£rom the respective holder sections 524, in interlock with the 
operation of the firee-lodc mechanisms 553. 

More precisely, when the instrument 520 is attached to the 
associated holder section 524, the &ee-lock mechanisn 553 
automatically locks the beiKling mechanism 537, whereby 
the drive force of the linear actuator 541 is reliably trans- 
mitted to the bending mechanism 537. When the connecting 
nut 521 is loosened to pull the instrument 520 from the body 
cavity, the free-lock mechanism 553 automatically unlocks 
the bending mechanism 537, regardless of the surgeon's 
intention. Tbe bending mechanism 537 can thereby be bent 
freely, so that the surgeon can remove the instrument 520 
from the body cavity, smoothly through the trocar 530. 

Should the surgeon fail to uidock tbe bending mechanisn 
537 to remove the instrument 520 from the body cavity, the 
mechanism 537 is automatically unlocked when the surgeon 
loosens the nut 521. Once unlocked, the bending mechanism 
537 can be bent fr^ly. The instrument 520 can therefore be 
pulled from the body cavity, snKxitbly through the trocar 
530, and the insertion section 520fr of the instrument 520 is 
not damaged at all. 

The surgeon need not be bothered to lock the bending 
mechanism 537 after attaching the instrument 520 to the 
holder section 524 or to xmlock the mechanism 637 before 
removing the instrument 520 frx>ni the body cavity. The 
same holds true of the operation of the 3D scope 523. Hence, 
with the surgical manipidator system according the nine- 
teenth embodiment it is possible for the surgeon to perform 
a transcutaneous endoscope surgery in high safety. 

The free-lock mechanism 553 can not only interrupt and 
resume the transmission of a drive force to the bending 
mechanism 537, but also change this drive force by tight- 
ening the connecting nut 521 to any desired degree. If the 
data representing a bending angle desirable for the mecha- 
nism 537 has been destroyed, the drive force is be reduced 
greatly so that the mechanism 537 may easily be bent with 
an external force. 

As has been described, the insertion section of either 
manipulator can be detached from the main section of the 
manipulator. Once detached from the main section, the 
insertion section can be sterilized with ease. Moreover, since 
some con^nents other than the insertion section can be 
removed from the main section of the manipulator, they can 
be easily sterilized. Further, once detached from tbe main 
section, these components can be stored in sterilized con- 
dition and can therefore be maintained in better conditions. 
The manipulators may interfere with each other in the course 
of surgery. The interference can be eliminated, merely by 
switching the insertion sections of the manipulators in terms 
of their positions. In brief, the nineteenth embodiment gives 
the surg^n great freedom of observing and treating the 
object of surgery, makes it easy to sterilize the insertion 
sections (e.g., an endoscope and a medical instrument), and 
can eliminate interference, if any, between the insertion 
sections easily and quickly. 

The twentieth embodiment of this invention will be 
described with reference to FIG. 60 and FIGS. 61Aand 61B. 
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This embodiment is a surgical manipulator system which 
is similar, as a whole, to the system of the nineteenth 
embodiment (FIG. 50). The components similar or identical 
to those of the nineteenth embodiment are designated in 

s FIGS. 60, 61A and 61B, and will not be explained in detail. 
This embodiment is diaractwizcd by the provision of two 
fastening devices for fastening a master manipulator and a 
surgical manipulator, respectively. As shown in FIG. 60, the 
first fastening device 608 is cormected to the proximal end 

10 of the master manipulator 501 and removably secured to the 
bedside rail 513 of the operating table 512. Similarly, the 
second fastening device 608 is connected to the proximal 
end of the surgical manipulator 503 and removably secured 
to the bedside rail 513. 

15 The fastening devices 608 are identical in structure. Only 
the first fastening device 608 connected to the surgical 
manipulator 503 will be described below, in detail. 

As shown in FIGS. 60, 61A and 61B, the first fastening 
device 608 comprises a clamp mechanism 617 and a main 

20 body 621. The main body 621 is coimected to the proximal 
end of the surgical man^ulator 503. Tbe damp mechanism 
617 comprises a stationary clomp 618, a movable damp 
619, and a fastening knob 620. Tlie stationary chmp 618 is 
connected to the main body 621. The damps 618 and 619 

25 have CTgagement member 618a and 619a, respectively. 
Tlie members 618a and 619a can contact the upper eiui and 
lower end of the bedside rail 513, respectively. The station- 
ary clamp 618 has a screw hole^ whereas the movable damp 
619 has a through hole. The fastening knob 620 has a screw. 

30 To secure tbe surgical manipulator 503 to the bedside rail 
513, the fastening knob 620 is turned in one direction, with 
the screw passing through the hole of the movable damp 
619, thereby driving the screw into the screw hole of the 
stationary clamp 618. The clamps 618 and 619 damp the 

35 bedside rail 513 between them, as shown in FIG. 61A. The 
surgical manipulator 503 is thereby fastened to the bedside 
rafl513. 

To remove the surgical manipulator 503 from the bedside 
rail 513, the fastening knob 620 is turned in the opposite 

40 direction, moving the movable clamp 619 away from the 
stationary clamp 618, as shown in FIG. 61B. Ibe fastening 
device 608 can then be removed, together with the manipu- 
lator 503, from the bedside rail 513. 

In preparation for surgery to be performed by using the 

45 surgical manipulator system, the surgical manipulator 503 
and the m^ter manipulator 501 are secured to the same side 
of the operating table 512 as shown in FIG. 50, by means of 
the first and second fastening devices 608. An observation 
manipulator 504 is secured to the operating table 512. A 

50 medical instrument 520 is attached to the surgical manipu- 
lator 503, and a 3D scope 523 to the observation manipulator. 
504. The instrument 520 and the 3D scope 523 are inserted 
into a body cavity as illustrated in FIG. 50. An image of the 
interior of the body cavity, provided by the 3D scope 523, is 

55 displayed by the HMD 510 the surgeon 513 wears and also 
by the TV monitor 511 located near the operating table 512. 

The surgeon 514 starts surgery on an object present in the 
body cavity. That is, he or she operates the master manqiu- 
lator 501, thereby remote-controlling the surgical manipu- 

60 lator 503, while observing the image of the interior of a body 
cavity, which is displayed by the HMD 510 or the TV 
monitor 511. 

As indicated above, the surgical manipulator 503 is fas- 
tened to tbe same side of the operating tabic 512 as the 
65 master manipulator 501. This means that the surgeon 516 
remains within a reach of the surgical manipulator 503 while 
operating the master manipulator 501. Should it become 



04/21/2004, EAST Version: 1.4.1 



5,876,: 

43 

necessary for the surgeon 516 to operate the surgical 
manipulator 503, as in emergency, he or she can immedi- 
ately start operating the surgical manipulator 503. Since the 
surgeon 516 stands by the patient 514 while while operating 
the master manipulator 501, the patient 514 feels much s 
relieved. 

The twenty-first embodiment of the invention will be 
described with reference to FIGS. 62 and 63. which show the 
support mechanisms for a master maDq)ulator 501 and a 
surgical man^>ulator 503. lo 

As shown in FIG. 62, a fastening device 608 of the same 
type used in the twentieth embodiment (FIG. 60) secures a 
master manipulator 501 to an operating table. To state more 
correctly, the vertical pillar 631 connected to the main body 
622 of the device 608 supports the horizontal body 632 of 15 
the master man^ulator 501. The pillar 631 can be moved 
vertically with respect to the device 608 by rotating a 
height-adjusting Imob 633 mounted on the main body 622 of 
the fastening device 608. 

As shown in FIGS. 62 and 63, the horizontal body 632 of 20 
the master man^julator 501 is mounted on a vertical shaft 
635. The vertical shaft 635 is supported by a bearing 639 and 
can rotate around its axis a,. Its angle of rotation is detected 
by an encoder 636 which is attached to the upper end of the 
vertical shaft 635. A horizontal member 637 is secured to the 25 
lower end of the vertical shaft 635 and is rotated around the 
axis a3 when the ^aft 635 rotates. A bearing 634 supports 
a horizontal shaft 638. An encoder 640 is connected to one 
end of the horizontal ^aft 638, for detecting the angle by 
which the horizontal member 637 and the horizontal shaft 30 
638 rotate relative to each other around an axis a^. A 
horizontal arm 641 is fastened to the horizontal shaft 638 by 
means of a pin 642. A vertical am 643 is siq)ported by a 
bearing 639. An encoder 644 is connected to one end of the 
vertical arm 643, for detecting an angle by which the 35 
horizontal shaft 638 and the vertical arm 643 rotate relative 
to each other around an axis a2. The encoders 636, 640 and 
644 are mounted on three bases 645, respectively. 

A vertical arm 646 is rotatably connected at its lower end 
to one end of the horizontal arm 641 suc^ that the arm 646 4o 
remains parallel to the vertical arm 643 at all times. A 
horizontal arm 647 is rotatably connected at one end to the 
upper end of the vertical arm 643 and at a middle portion to 
the upper end of the vertical arm 646. The vertical arms 643 
and 646 and the horizontal arms 641 and 647 constitute a 45 
parallel link. A weight 648 is connected to one end of the 
horizontal arm 641, and a weight 648 to one end of the 
vertical arm 643. These weights 648 serve to secure gravity 
balance among the arms 641, 643, 646 and 647. 

As shown in HG. 62, a handle section 649 is removably 50 
connected to the distal end of the horizontal arm 647. The 
handle section 649 has three axes 650, 651 and 652. An arm 
X, an arm Y and an arm Z can rotate around these axes 650, 
651 and 652, respectively. The angles a^, aj and a^ through 
which the arm X, the arm Y and the arm Z are rotated can 55 
be detected by encoders 653, 654 and 655. The angle by 
which a handle 656 rotates can be detected by an encoder 
657. The information representing the angles of rotations, 
which the encoders 636, 640, 644, 653, 654, 655 and 657, is 
supplied to a motor control unit 502. 60 

The motor control unit 502 calculates the position and 
orientation of the handle section 649 from the information, 
the lengths of the vertical arms 643 and 646 and the lengths 
of the horizontal arm 647. The unit 502 controls the surgical 
manipulator 503 in accordance with the position and orien- 65 
lation of the handle section 649 and the angle of rotation of 
the handle 656 detected by the encoder 657. More precisely, 
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the unit 502 moves the vertical support of the manipulator 
502 in the direction of arrow Aj, rotates the vertical support 
in the direction of arrow A^, and moves the horizontal arm 
of the man^ulator 503 in the direction of arrow Aj. Further, 
the unit 502 rotates the insertion section 659 of the manipu- 
lator 503 in the direction of arrow A^, bends the distal end 
portion of the manipulator 503 in the direction of arrow A^ 
or A^, and open or closes the instrument 658 attached to the 
distal end of the man^ulator 503. The horizontal arm and 
the insertion section 659 are connected, end to end, by a free 
joint 660. 

When the surgeon 516 holds and moves the handle 656 in 
any direction in a space, the encoders detect the angles of 
rotation around the axes a^ to a^. From these angles of 
rotation and the lengths of the arms 643, 646 and 647, the 
motor control unit 502 calculates the position and orienta- 
tion of the handle section 649. In accordance with the 
position and orientation of the section 649, the unit 502 
remote-controls the surgical manipulator SQ3, making the 
manipulator 503 assume the position corresponding to that 
of the handle section 649 and the same orientation as that of 
the handle section 649. When the surgeon 516 opens or 
closes the handle 656, the encoder 657 detects the angle of 
rotation of the handle 656, and the motor control unit 502 
controls the surgical manipulator 503 such that the instru- 
ment 658 is opened or dosed by the same angle of rotation. 

The master manipulator 501 of the twenty-first embodi- 
ment comprises am^ supported and connected by rotary 
mechanisms, other arms constituting a parallel link, and 
encoders for detecting angles by which various parts are 
rotated around various axes. The position and orientation of 
the surgeon's hand are determined from the angles of 
rotation detected by the rotary encoders. Therefore, this 
embodiment is advantageous in the following respects: 

a) The master manipulator is compact enough not to lower 
the efficiency of surgery. All movable parts of the 
master man^mlator are rotatably supported, and no 
linear drive elements sudi as ball screws are required to 
drive the movable parts. The master manipulator can 
therefore be operated quite smoothly. 

b) Hie parallel link enables the surgeon to move the 
handle in a relatively large space, whereby the tool 
center point (TCP) of the surgical manipulator can be 
moved in as large a space. 

c) The position and orientation of the surgeon's hand are 
detected by rotary encoders wbidi are not influenced by 
vibration. The system can be used along with any other 
surgical system that generates much vibration, and can 
therefore be versatile. Since the surgeon only needs to 
manipulate the handle to perform surgery, he or she 
need not make any complicated preparation or acquire 
any special skill. This helps to shorten the operation 
time and lighten the surgeon's burden. 

The twenty-second embodiment of the invention will be 
described with reference to FIGS. 64 to 66 and FIGS. 67A 
and 67B. 

As shown in FIG. 64, the surgical manipulator 701 of this 
embodiment comprises a manipulator body 703 and a 
straight insertion section 702. Tlie manipulator body 703 is 
a multi-joint arm and has a point-locking mechanism and a 
position adjtisting mechanism, both designed to position the 
insertion section 702. Tlie insertion section 702 is coimccted 
at its proximal end to the distal end of the manipulator body 
703. The section 702 is thin enough to be inserted into a 
body cavity through an opening 708 incised in the body wall 
790. An endoscope 704, used as an end effector, is attached 
to the distal end of the insertion section 702. Although not 
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showD in FIG. 64, the twenty-second embodiment has mal side of the point-locking mechanism. The position 

another surgical manipulator which is identical to the adjusting mechanism is a support section 711 shown in FIG. 

manipulator 701. A medical instrument 705, used as an end 64. The section 711 can rotate around its axis (in the 

effector, is attached to the distal end of the insertion section direction of arrow CI in FIG. 64) and which can move up 

of the other surgical manipulator. What kind of an end 5 and down (in the direction of arrow C2). The section 711 

effector should be attached to the either surgical manipulator supports the shaft 710, which can be rotate (in the direction 

depends on which type of surgery is to be performed. of arrow Al) and back and forth (in the direction of arrow 

As shown in FIG. 64, only one bent part is provided C3). Therefore, when the support section 711 is moved 

between the insertion section 702 and the endoscope 704. vertically and the shaft 710 is moved linearly with reject to 

The same is true of the insertion section 702 of the other 10 the section 711, the position and orientation of the stationary 

surgical manipulator and the medical instrument 705. The bar 712 is changed. Ultimately, the point P defined above is 

endoscope 704 and the instnmient 705 are much shorter than moved. Needless to say, once the position adjusting media- 

the insertion section 702 of either surgical instrument. Hie nism and the second link 709' are locked after the point P has 

endoscope 704 has an illumination unit and an observation been shifted to a desired position, the point P remains at that 

unit. The medical instrument 705 has a forceps 706 for 15 position even if the second link 709' is rotated in the 

holding tissues, for peeling tissues off an oigan, effecting direction of arrow A2 and the stationary bar 712 is rotated 

suture or holding a needle. around its axis. 

The number of axes the manipulator 701 has is deter- Hence, once the position adjusting mechanism has been 

mined by the degree of freedom needed in order to set the so adjusted that the insertion section 702 is set in a ^ecific 

end effecter in any desired position and any desired 20 position in the opening 708» the insertion section 702 

orientation, and also by the position of the opening 708 remains at this position even if the position ac^ustingmedia- 

which is incised in the body wall 790. nism is moved. Thus, when the position a(^ting media- 

Gcnerally, the manipulator body 703 must have six nism is operated, with the point Pbcated at the opening 708, 
degrees of freedom in order to observe or treat an organ the TCP of the insertion section 702 can be freely moved in 
located at a given position in the body cavity. Since the 25 the body cavity without exerting any excessive force to the 
image obtained by the endoscope 704 is rotated by means of body wall 790. Since the point-lock mechanism prevents the 
image processing, however, five degrees of fieedom are insertion section 702 from moving in the opening 708 
sufBcient for the manipulator body 703. By contrast, the incised in the body wall 790 when the manipulator 701 s 
man^)ulator body of the other surgical manipulator (not operated, the insertioai section 702 need not have three 
show) having the medical instrument 705 as end effecter, 30 degrees of freedom. It suffices for the surgical manipulator 
needs to have six degrees of freedom. As for the position of 701 to have five axes in the case where the endoscope 704 
the opening 708, it is required that the TCP of the insertion is used as the end effecter, or six axes in the case where the 
section 702 of either surgical manq>ulator be maintained at medical instrument 705 is used as the end effecter. 
a specific position with respect to the opening 708 through- The medianism for driving the insertion section 702 with 
out the surgery. Otherwise, the insertion section 702 would 35 the endoscope 704 used as the end effecter will be described, 
apply an excessive force on the body wall 790. To this end, with reference to FIGS. 67A and 67B. 
the insertion section 702 must be held immovable in three As ^own in FIG. 67A, the insertion section 702 corn- 
axes, and accordingly needs to have three degrees of free- prises a movable cylinder 707, a guide cylinder 724 and a 
dom. rotary cylinder 725, The cylimlw 707 can move back and 

Therefore, the surgical manipulator 701 having the eodo- 40 forth. The guide cylinder 724 supports and guides the 

scope 704 must have eight degrees of fr^dom, whereas the movable cylinder 707. The rotary cylinder 725 supports the 

surgical manipulator (not shown) having the instrument 705 guide cylinder 724 and can rotate in the direction of arrow 

must have nine degrees of freedom. It follows that the A4. The rotary cylinder 725 has the above-mentioned bar 

manipulator 701 needs to have eight axes, and the manipu- 721 and supported by a proximal section 791 which moves 

lator (not shown) nine axes. Nonetheless, the surgical 45 in the same way as the bar 721. 

manipulator 701 shown in FIG. 64 may require but only five A pulley 717 is fastened to a member which supports the 

axes, by virtue of the provision of a point-locking media- endoscope 704 and which can be moved to rotate the 

nism illustrated in FIG. 66, endoscope 704 in the direction of anow A5. A wire 718 is 

As shown in FIGS. 66A and 66B. the point-locking wrapped around the pulley 717. The ends of the wire 718 are 

mechanism is comprised of six bars which are rotatably so fastened to two ball screws 719a and 719b, which are 

linked by six fulcrum pins, forming two parallelogrammatic rotatably supported in the rotary cylinder 725. The ball 

links 709 and 709! which are contrapositive to eadi other. screws 719fl and 7196 are rotated by electric motors 720fl 

The first parallelogrammatic link 709 is the disUl part of the and 720b, respectively. The movable cylinder 707 is biased 

point-locking mechanism, and the second parallelogram- toward the distal end of the insertion section 702, by a spring 

matic link 709' is the proximal part thereof, A shaft 710 is 55 732 always applying a tension on the wire 718. 

connected to the bar 712 which is stationary and one of the When the motors 720a and 720b are driven, the ball 

four bars constituting the second link 709'. The point- screws 719a and 719b pull the re^)ective ends of the wire 

locking mechanism can therefore be rotated around the axis 718 to the right Subsequently, the movable cylinder 707 is 

of the bar 712 when the shaft 710 is rotated. Point P al which moved to the right for half the sum of the distances the ends 

the axis of the stationary bar 712 intersects with that of the 60 of the wire 718 have been pulled. The pulley 717 is thereby 

bar 721 of the first link 709 does not move at all, no matter rotated in the direction of arrow A5 by a circumferential 

how the six bars arc moved in the same plane as the distance equal to the difference between the distances the 
manipulator body 703 swings and . how much the point- ends of the wire 718 have been pulled to the right, 

locking mechanism is rotated around the axis of the station- The rotary cylinder 725 is supported by a bearing 722 and 

ary bar 712, 65 connected to an electric motor 720c by gears 723. The guide 

As mentioned above, the manipulator body 703 has a cylinder 724 is fastened to the rotary cylinder 725 by bolts 
position adjusting mechanism which is located at the proxi- 726. When the bolts 726 are unscrewed and the wire 718 is 
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disconnected from the ball screws 719a and 719b, the 
movable cylinder 707 and the guide cylinder 724, both to be 
inserted into a body cavity, can be removed from the 
proximal portion of the insertion section 702, as is Ohistratcd 
in FIG. 67B. The movable cylinder 707 and the guide 5 
cylinder 724 can then be washed and sterilized. 

The components of the surgical manipulator 701 shown in 
FIG. 64 are linearly moved or rotated in the directions of 
arrows Al to A6 by means of actuators and transmission 
members. The actuators arc driven in accordance with lo 
commands generated by a controller (not shown) electrically 
connected to the suigical manqiulator 701. The controller 
can generate commands by playback method and master- 
slave method. In the playback method, the controller gen- 
erates commands from the operation patterns already pro- 15 
grammed and stored. In the master-slave method, the 
controller generates commaisds as a surgeon operates a 
master manipulator 714 shown in FIG. 65. The master 
manipulator 714 has a multi-joint mechanism 715 and 
encoders 716. The mechanism 715 has as many degrees of 20 
freedom as the surgical manqiulator 701. The encoders are 
provided at the respective joints of the mechanism 715. 

As explained above, the surgical manipulator 701 has five 
degrees of ficeedom (arrows Al to AS) when the endoscope 
704 is used as end effecter, and six degrees of ficeedom 25 
(arrows Al to A6) when the medical instrument 706 is used 
as end effecter. The point-lock mechanism and the position 
adjusting medianism cooperate to set the TCP of the inser- 
tion section 702 at a desired position in a body cavity. More 
precisely, the components of the point-lock mechanism are 30 
moved in the directions of arrows CI, C2 and C3, thereby 
setting the point P in the opening 708, and the components 
of the position adjusting mechanism are moved in the 
directions of arrows Al and A2, thereby setting the TCP of 
the insertion section 702 at the desired position in the body 35 
cavity. 

Hie TCP of the insertion section 702 is then moved to a 
target object in the body cavity, by moving the components 
of the ins^on section 702 in the directions of arrows A3, 
A4 and A5. Namely, the TCP of the end effecter attached to 40 
the distal end of the movable cylinder 707 is brought to the 
target object as the cylinder 707 is moved in the direction of 
arrow A3, the rotary cylinder 725 is rotated in the direction 
of arrow A4, and the end effecter is rotated in the direction 
of arrow A5. Since the movable cylinder 707 is straight and 45 
thin, it approaches the target object &om the opening 708 in 
a straigbi path, not contacting any other object present in the 
body cavity. The end effecter is rotated to have its orientation 
adjusted, thus having its TCP located exactty at the target 
object. The end effecter does not contact any other object, 50 
cither, while being rotated. This is because it is far shorter 
than the movable cylinder 707. 

As has been described, the surgical manipulator 701 is so 
designed that the TCP of the end effecter coupled to the 
insertion section 702 can move in a large space in a body 5S 
cavity, and that neither the insertion section nor the end 
effecter conUcts any object other than the target object. In 
other words, the manipulator 701 has many degrees of 
freedom, applying no excessive force on the target object 
and enabling the end effecter to treat the target object or 60 
provide the surgeon with an image of the target object. It is 
desired that the end effecter have a length one-fifth or less 
the length of the movable cylinder 707, as measured from 
the center of the pulley 717. 

The twenty -third embodiment of the invention will be 65 
described with reference to FIG. 68 and FIGS, 69Ato 69D. 
FIG. 68 shows a surgical manipulator 701d which is 
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designed to extract the kidney 760. The main body of the 
manipulator 701d has the same structure as that of the 
surgical manipulator used in the first embodiment. 
Alternatively, the main body may have a different stmcture. 

The surgical manipulator 701d comprises a straight inser- 
tion section 734, an expander 735, a tissue crusher 736, a 3D 
endoscope 737, micro-manipulators 738, and two-armed 
micro-manipulators 770. The expander 735 is nrtounted on 
the distal end of the insertion section 734. It is constructed 
like a parasol, having a phirality of nbs 735a and a flexible 
cover connected to the ribs 735a, and can be opened after 
inserted into a body cavity. The tissue crusher 736 and the 
3D endoscope 737 are connected to the expander 735 and 
located inside thereof. The micro-mampiilators 738 are 
connected to the distal end of the insertion section 734, for 
holding an ablation forceps and a pusber-renoover. The 
two-armed micro-manipulators 770 are provided to apply a 
suture. Atouch sensor 739 is mounted on the holding surface 
of the first two-armed micro-manipulator 770. The kidney 
760 is severed from (he urethra and the kidney artery is too 
large to pass through an opening 708 incised in the body 
walL To extract the kidney 760 outside the body wall, the 
expander 735 is closed, wrapping up the kidney 760, and the 
tissue crusher 736 is driven, crushing the kidney 760. the 
tissue crusher 736 has an ultrasonic oscillator for crushing 
the kidney 760 and a suction device for extracting the 
crushed tissue. 

FIGS. 69A to 69£ show how the two-armed micro- 
manipulators 770 are operated to tie up the kidney artery 763 
before the kidney 760 is severed from the kidney artery 763. 
First, the micro-man^ulators 770 are operated to wrap a 
suture 740 around the artery 763 as illustrated in FIGS. 69A 
and 69B. Then, they are operated to tighten the suture 740, 
tying up the kidniey artery 763, as shown in FIGS. 69C and 
69D. The two-armed micro-manipulators 770 automatically 
conduct this sequence of cumbersome operations in accor- 
dance with a prepared program. 

In the twenty-third embodiment, the ciqpander 735 
expands the body cavity and protects the inner wall thereof, 
without performing a pneumoperitoneal operation. Further, 
the toudi sensor 739 serves to protect the kidney 760 against 
damage. Automatically controlled, the micro-manipulators 
738 and 770 scarcely contact any organ than the kidney 760. 
SbouM they be moved toward other organs, they are stopped 
by the expander 735. No force is applied onto the other 
organ while the surgical manipulator lOld is functioning. 

As has been described, with the twenty-second and 
twenty-third embodiments it is possible to set the insertion 
section at a desired position, next to move the insertion 
section straight to a target object present in a body cavity, 
then to orientate the end effecter toward the object. As a 
result, the target object can be observed clearly or treated 
readily. Since the insertion section need not be bent in a 
complex manner, it does not damage any object other than 
the target one. Furthermore, since the end effecter is far 
shorter than the insertion section, it can treat the target object 
only and obtains an image of the target object only. 

The 3D position sensor used in the embodiments 
described above will be explained, with reference to FIGS. 
70A to 70E. 

FIG. 70A shows an electromotive pickup sensor which 
can be used as the 3D position sensor. This pickup sensor has 
a permanent magnet 800 ar>d a pair of coils 801 and 802 
sandwiching the magnet 800. The second coil 802 moves 
across the magnetic field of the magnet 800 and generates an 
electromotive force which is proportional to the speed at 
which it moves. The output of the pickup sensor is propor- 
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donal to the speed of the coil 802. The first coil 801 is used 
to achieve electromagnetic braking. The coils 801 and 802 
function as weights. The output of this pidcup sensor must 
be electrically integrated to detect the position of an object. 
To delect a 3D position of the object, three electromotive 
pickup sensors need to be combined, with their detection 
axes a (Le., detected vectors) intersect at right angles to one 
another as shown in FIG. 700. 

FIG. 70B shows a variable-Tesistance pickup sensor 
which can be used as the 3D position sensor. This sensor is 
designed to detect the strain on a strain gauge or a semi- 
conductor strain gauge, in the form of a change in electrica] 
resistance. More precisely, the sensor of FIG. 70B is an 
acceleration pickup which have four semiconductor gauges 
805, one weight M and a DC bridge circuit The DC bridge 
circuit detects the changes in the resistances of the gauges 
805, which are proportional to the displacement of the 
weight M. The ou^ut of the DC bridge circuit, which is 
acceleration, must be electrically integrated twice in order to 
detect the position of an object. To detect a 3D position of 20 
the object, three variable-resistance pickup sensors need to 
be combined, with their detection axes a (Le., detected 
vectors) intersecting at right angles to one another as shown 
in FIG. 70D. 

FIG. 70C shows a piezoelectric pickup sensor whidi can 25 
be used as the 3D position sensor, too. The piezoelectric 
pickup sensor has a hollow cylindrical weight M, a piezo- 
electric element 806 and a post 807. The element 806 is 
mounted on the post 807 and secured thereto. The weight M 
is secured to the upper and lower ends of the post 807, and 30 
concealing the piezoelectric element 806. When applied 
with a stress proportional to acceleration, the element 806 
accumulates an electrical diarge proportional to the stress. 
Thus, the diaige corresponds to the acoeleratioo. The output 
of the piezoelectric pickup sensor must be electrically inte- 
grated twice in order to detect the position of an object. To 
detect a 3D position of the object, three piezoelectric pickup 
sensors need to be combined, with their detection axes a 
(i.e., detected vectors) intersecting at right angles to one 
another as shown in FIG. 70D. 40 

A servo acceleration pickup can be as the 3D position 
sensor, as well. This pickup sensor has a weight and a servo 
anq)Mer. Tlie displacement of the weight is detected, and a 
current proportional to the displacement is fed back to the 
serve amplifier, thereby generating an electromotive force 45 
proportional to the displacement of the weight The force is 
applied to the weight to sensor the position of the weight. 
TTie feedback current is proportional to the acceleration. The 
output of the servo acceleration pickup sensor must be 
electrically integrated twice in order to detect the position of 50 
an object. To detect a 3D position of the object, three servo 
acceleration pickup sensors need to be combined, with their 
detection axes a (i.e., detected vectors) intersecting at right 
angles to one another as shown in FIG. 70D. 

FIG. 70E shows an oscillation gyroscope which can be 
used as the 3D position sensor This sensor functions, uti- 
lizing the dynamic phenomenon that an oscillating object 
generates a Coriolis force when applied with an angular 
velocity. In other words, the sensor acts like the well-known 
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Fourcault pendulum. As shown in FIG. 70E, a piezoelectric 60 clamp forceps 



809/) extending in the Y axis detects the angular velocity Qq. 
The angular velocity Qg is integrated, obtaining an angle. 
The oscillation gyroscope may be attached to the surgeon, 
for example, to his or her head, to detect the orientation of 
the head. 

Additional advantages and modifications will readily 
occur to those skilled in the art. Therefore, the invention in 
its broader aspects is not limited to the specific details, and 
representative devices, shown and described herein. 
Accordingly, various modifications may be made without 
departing fh>m the spirit or scope of the general inventive 
concept as defined by the appended claims and their equiva- 
lents. 

What is claimed is: 

1. A surgical manipulator system comprising: 

at least one surgical manipulator whidi holds a surgical 
device for performing a desired operation; 

at least one operating means for oontrollii^ said surgical 
manipulator; 

detecting means for detecting one of the following: 

(a) a geometrical relationship between said at least one 
surgical manipulator and said at least one operating 
means, and 

(b) a geometrical relationship between said at least one 
surgical manipulator and another surgical manipula- 
tor; and 

drive control means for controlling said at least one 
surgical manipulator in response to said operating 
means and said detecting means such that said surgical 
device p^orms a desired operation. 

2. The system according to claim 1, wherein said detect- 
ing means comprises: 

a first position detecting means for detecting at least one 
of a position and orientation of said at least one 
operating means by using a first base coordinate system 
as a reference; 

a second position detecting means for detecting at least 
one of a position and orientation of said at least one 
surgical manipulator by using a second base coordinate 
system as a reference; and 

a coordinate-system conversion means for converting the 
coordinate system of one of said operating means and 
said surgical man^idator, to the coordinate system of 
the other of said operating means and said surgical 
manipulator, so as to operate said operating means and 
said surgical manipulator in a common coordinate 
system. 

3. The system according to daim 2, 

wherein said second position detecting means includes a 
coordinate -system conversion means for converting the 
first base coordinate system to the second base coor- 
dinate system. 

4. The system according to daim 1, herein said surgical 
device is one selected from the group consisting of an 
endoscope, an ultrasonic echo probe, a surgical microscope, 
a holding forceps, a surgical knife, a suture applicator, a 
syringe, an ablation forceps, a needle-holding forceps and a 



device 809 comprising a detecting element 809a and a 
driving clement 8096 is adhered to a tuning fork or an 
oscillator 810 shaped like a tuning fork. Angular velocity Qq 
is applied to the oscillator 810 which is oscillating. Then, the 
oscillator 810 generates Coriolis forces acting in the direc- 
tion of oscillation (K axis) and in the direction (Y axis) 
perpendicular to the direction of oscillation. The element 
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5. The system according to claim 1, wherein said at least 
one operating means is one selected from the group con- 
sisting of a master manipulator, a three-dimensional position 
sensor, an eye-axis detector which designates a position on 
the line of the eye axis, a joystick, a speech recognition unit 
which designates a geometrical relationship to said at least 
one surgical manipulator and a gyroscope. 
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6. The system according to claim 1, 

wherein said at least one operating means includes two 
operating means, and 

wherein said system further comprises calculating means 
for calculating first tool center points corresponding to ^ 
a first operating means and second tool center points 
corresponding to a second operating means, said cal- 
culating means determining a difference between the 
first and second tool center points in order to operate 
said surgical device. 

7. Hie system according to claim 6, wherein one of said 
two operating means is a master manipulator, and the other 
of said two operating means is a head-mounted di^lay. 

8. The system according to claim 6, wherein said drive 
control means controls said at least one surgical manipulator 
only when said difference between the first and second tool 
center points is within a predetermined range. 

9. The system according to claim 8, wherein said drive 
control means stops said at least one surgical manq}ulator 
only when said difference between the first and second tool ^ 
center points is outside the predetermined range. 

10. The system according to claim 1, wherein said at least 
one surgical manipulator includes first and second surgical 
manq)ulatOTS, and said detecting means comprises a perma- 
nent magaet attached to said first surgical manipulator and ^ 
a HaU element attached to said second surgical manipulator. 

U. The system according to claim 10, wherein said first 
surgical manipulator has an endoscope. 

12. The system according to claim 1, herein said at least 
one i^rgical manipulator comprises: 

a first manipulator imit which holds first surgical means 

for treating an affected part of a subject; and 
a second man^ulator unit which holds a second surgical 

device for providing an image of the affected part; 35 
i^erein said at least one operating means comprises: 
a master manq}iilator for operating said first manipu- 
lator tmit; and 
a sensor for guiding said second manipulator unit; 
v^erein said surgical manipulator system further com- 40 
prises display means for displaying the image pro- 
vided by said second surgical device; and 
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wherein said sensor is adapted to be secured to a 
surgeon's head, and said second manqjulator \mit is 
moved in the same way as said sensor. 

13. The system according to claim 1, further con:^rising 
a joint mechanism for connecting said surgical device to said 
surgical manq)ulator, said joint mechanism having a free 
rotational axis and at least one degree of freedom. 

14. The system according to claim 1, further comprising 
a fastener means for removably fastening said surgical 
manipulator to an operating table. 

15. Tbe system according to claim 1, wherein: 

said surgical device comprises a straight insertion section 
and a surgical section which is connected to a distal end 
of the insertion section and which is able to bend to: 

(a) treat an affected part of a subject and 

(b) provide an image of tbe affected part of the subject, 
and 

said stirgical manipulator comprises a coupling device for 
holding said insertion section, while allowing said 
insertion section to move back and forth in a longitu- 
dinal direction of the insertion section, and a position 
device for positioning said insertion section. 

16. The system according to claim 1, further oonq}rising 
a cormecting mechanism for removably cormecting said 
surgical device to said surgical manqnilator. 

17. The system according to claim 16, further oonq>rising 
an operation section for bending said surgical device, and 
coupling means for controlling a magnitude of a force to be 
transmitted from said operation section to said surgical 
device, such thai said surgical device is flexed without 
restriction. 

18. The system according to claim 1, further comprising 
an operation section for bending said surgical device, and 
coupling means for controlling a magnitude of a force to be 
transmitted from said operation section to said surgical 
device, such that said surgical device is flexed without 
restriction. 

» « « * * 
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